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INTRODUCT19N ^ 



Over\&iew of Project Dfevelopmentai Continuity (PDC) 

_. _^ _ _ _ _ _ ._ . _ ^ 

The Office of Child Development^ originated Project ^ 
Developmental Continuity (PDC) in 4974 as a Head Start 
demonstration program "aimed at promoting grfeater continuity 
of educatioh and comprehensive child development services 
for children as they make the transition from preschool to 
school." The single most important effect of this under- 
► taking^ it is hoped, will be to -enhance the social competence 
of the children served — that is^ to increase their everyday 
effectiveness in dealing with their environment (at school , 
at home^ in the community, and in society)* PDC alSo aims 
to bring about broader and more intensive involvement of 
parents and teachers in the governance of school affairs , 
and to promote positive change in the institutional process^ 
even beyond the people who may occupy the institution at a 
given time. 

As part of the overall Head Start improvement and 
innovation effort^ PDC .Emphasizes the involvement of adrpini- 
strators, classroom staffs and parents in formulating educa-- 
tional gdals and developing a comprehensive curriculum. The 
object, is to ensure that children receive continuous 
individualized attention as. they progress from 1 Head Start 
through the early primary grades. If the program ia successful ^ 
existing discontinuities between Head Start and elementary 
school experiences will be reduced by PDC mechanisms that 
encourage communication and mutual decision^making among 
preschool and elementary school teachers^ administrators, 
and parents * . 

Two program models provide alternative ways of establishing 
the administrative structure for continuity. In the Preschpol- 
School Linkac7Gs approach, administratively separate Head Start 
and alementary school programs are brought together by the 
device oi a PDC Council, whose membership includes teachers, 
parents', and administrators from botK organizations , In the 
Early Childhood Schoolsvapproach , Head Start and elementary; 



school p.rograms 'a're combined both administratively (by the * 
Council) and physically (in the same ^ building) , creating a " 
new institution., In both ^appr6ackis a qualitatively different 
• program is expected to einerge as a result^ of. Hfead Start'^ ^ 
elementary - school cooperation. " . ' 

•.^ Continuity i^ eKpected to be e^tablish'ed in^'twd contexts'i 
that of "'fehe , individual child and that of the school structure* 
In the . f4i:.st\con text , ^ coniinuity hte'an for example, that a". . 
child should' ho t'^ have to have.^his o^her personar nature =and - 
needs rediscovered each year as he or she moves from one ^ade 
to the next \nstead the child shouLfi become a iTtore fully 
recognize$3 . member 5f the school "family- as time passes. 
In the context of school structure^ continuity In^lies' a 
cooperative pursuit of common goals, and this involves artT^ 
culaticm of philosophies and methods in all the various, areas 
of school enterpri&e-. It is expected that structural continuity 
will conjfcribute directly to continuity in the attention given 
to individual children* ' 

School 'organizations at^4!5 sites around, the country 
received OCD funding during fl974--75 (E'Togram Year I) to^design 
and . plan future implementation of seven prescribed components 
of PDC : --^ > \ . - . ^ f ' 

■ • ' " ' ' . ■ ■■ ^ . 

^. Administration : administrative coordination between' .< 

and within ^Head Start aAd elementary^schopl ; - . . 

0 Education: coordination of curriculum approaches^- i 



and educational go^ls;^ 



o Training:.. preserYice and inservice training for 
teachers and childrearing training for parents ; 

o Developmenta l Sup port Serv ices: comprehensive 
services (medical ^ nutritional/ and social) toj 
children and families; ^ 

G Parent Involvement t parent participation in policy^ 
making / home^schoO;! activities/ and classroom 
visits or volunteering; 

® Ser vices f or the Ilandicapped : services for handicapped; 
children, and children with learning disabilities; 

. p 

o Bilring ual./Si cultural and M ulticultural Education : 
programs for bil ingual/bicul tural or mui ticultural 
child:ren. 



During Year II, 1975-76, 14 sites ' (erne had!t?^^^thdrawn:^ 
■'voluntarilf") / comprising a total 4 2 ^Head St ato^.*^.- centers 
■ arid elementary sa^ools, began their, "start-up" -^43.^ ^ pilot-^ 
testing - their adaptations of th^ PDC prpgnm ad^rding^ to 
-plans -they had drawn^ lip. during Program Year I . ..^'^'tf^^e prpgram 
is gurrently in Year III, 1976-^77^^ and PDC is nb.WjsuppOBed 
to e>^isi in mature form at the 11 participating- ]|;ites (a ^ 
second^ite withdrew at the end of Year II). I^^;;^ ' longitudinal 
, sti/dy of PDC IS commissioned/ the- prqgrpm * s ef f e'(|^;£s will .be 

e^mined throughout -/the' period beginning with present * 

_..^^ar and continuing until -dhe end of the 1980--8b^bchool year. 
This is thq period during wliich children in the ^terent 
testing samples (Cohort 2) will progress from He^.'d Start 
though grade 3, , ' 



■ ^ ■ 



Purposes of the PDC Evaruat ion 

The major purpose of the PDC evaluation is to kid the 
' Office of Child pevelopment, in its efforts to design effective 
programs for early childhood education. To acconiplish this, 
the . evaluation^will. ultimately have to provide answers to. 
the 'following critical ^questions about PDC ' s impact: ' ' ^ 

a How does PDC affect childre|i-s social competence? 

m How does PDC affect parents? ^ ^ . 

m How does PDC affect the attitudes and workstyles 
of .teachers and other staffs? , ■ 

e m How does PDC aftect the school organization in 

terms of .philosophy , methods and social clim.ate? 
> , ■ \. . " ■ 

^ V In addition to describing the consequences of PDC* the 
■evaluation will describe and analyze thffe processes, that Imd . ^ 
to those consequences* Although the assessfnent of children's' 
social competence*, is very important and i& emphasized in the 
present-: report , "^the relationship of this to thelrq^t.;.of the^ | 
evaluation' should not be neglected. , Volume 2 of Interim 
Report IV (August ^976 ) delineates^ the process evaluation . 
.more fully; it rs^^uff icreht to ., emphasize^ here that 'the aims 
of the total evaluation are to produce i conclusions about • 
what hap^^ned (Impact) and how and why ' it happened (pro'iess) . 
This infbrmation^will facilitate future decisions about ^ ■ 
whether wW progrnm' should be replloahnd, and if so, how ^ 
"i icaLlon can beyb be acconip 1 i s hedr in the^ light of past 
experience , ' ..\ , = . 



Purposes of this Report 

Last year (Program Year^ j^/) a sample of Head Start 
children participated in pil6t-testing of the instruments % 
tentatively selected for the PDC battery. Demographic 
data were also collected for t^ese children, information 
regarding grade-to--grade attrition for children in past 
■ years was gathered from eaph site. This information provided' 
-^preliminary answers to three critical questions= - 

" ^ ' • " ^ 'J . / - , ■ ' : 

1. Are. the measuring instruments appropriate to 
the task? ^ 

' ^ ' ' ' ' " J 

2. ^ Are the PDC and comparison groups really/ 

colnparable?^ . ' ^' ■ 

3. will' large enough samples -o^ children remain 
in PDC and comparison schools, at each site to 
permit a Jongitudiilal study, of program effects? 

On the basis, of answers gained last year to question 1^ . 
some instruments^ were. eliminated from 'the battery and- others 
were modified. On the basis of last year* s findings with ^ 
regard to question 2, recommendations^'were made to sites 
concerning procedures that could be followed during the 
Head Start enrdllment period to balance PDC and comparison 
groups. in certarn important respects. And on the basis 
of the 'pro jections made in connection with question 3\ 'the 
likelihood Vas assessed that the samples would remain' p. • 
large enough over a five-year term to permit continuing 
stcitistdcal analysis of their performance.^ 

IPhese same issues'---instruinent suitability , comparability-^' 
of groups, and adequacy of sample size--are examined again 
in this repprtMjsinq dat^aV gathered In the fall of Year III. 

he purfDose of this re^^Kaminat ion is.v'tp ■ verify preliminary ^ 
iiVdicdtions from-'^Year I^,.. These findings and others are ' 
i/itegrated in this report',- and their implicationsft Tor a 
ohgitudinal study of PDC's impact are consi^^red. The '\ 
iscussioh Goncludes with g^<^li mi nary, recommendations for 
the futu.re\,of the evaluat^Cn. ' ' ^ / ^ 

tallowing are brie J statements of .the rationales for 
_^,^dressing. the pa^ticula^ issues dealt with in this report, , 




-These conclusions are documented arid discussed in Interim 
Report IV, V olume 1, Pilot Year Impa ct Stud y: Instrument 
Ch aracteristic s and' Att rition Trends August 1976,^ 
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Instrument appropf lateness ■ sTmce the ultimate'" goal 
of PDC is to enhance the social compajtance of. children, it 
is essential to the evaluation thatT'che instruments used 
yield measures that, collectively/^'^ represent social competence 
in an accurate and meaniJ^gf ul way. The criteria ufeed in this 
and past reports fpr^ judging the adequacy of the instruments 
include, ^ . . - - . 

m internal consi|Stency (the extent to which an 
instrument ' s fc onstituent items "agree on what 
they reflect 1 about an indivi^dual , ^ 

'1 ' " ' . ^* / ^ 

© stability (correspondence \between total scores on 
the same instrument given to. the same children at 
^ .. two time-^points) , r. 

© vali'di ty (the degree to which ah instrument measures 
what it ds, supposed to measure) , f . 

■ • ' ■ ■ 1 

Q . sensitivity to change (an instrument's responsiv^ne^ 
to change over time in' the characteristics measuj 

m relevance to social competence (the degree to vmich 

the information produced by an instrument contributes . 
to knowledqe of a child's social competence) ^ 

© -He ^elopfaental span (su^itability for children across 
;the age range prospectively covered by the PDC 

\ evaluation)^ and / - / ' ^ 

* ' "' -. ' 

m - easp o f admin istration (the cost in'time^ effort^ 
and resources of administerihg an .instrument ) ^ . ' 

■ C omparability of groups . The effects of PDC upon 
children will be determined primarily by comparing the perfor-/^ 
mance of children in^&DC testing samples with the performance 
of children who are similar^ ,but who do not participate ^ in 
PDC ii.e.f, a comparison group).. The assumptiO'n implicit _ in 
this is that. the children in the two groups would remain 
parallel were it notl^for the intervention of PDC^ and ^hus 
the way children in the comparison group perform in ^the future 
stands for the way. PDC children would have performed without 
t hp [^rosumod advan taqe of PDC\ Whether this assumption 
itself 3t:ands or falls depends upon the initial eqpa valence 
fjf the two groups. Unless they are very similar^^ be-qin - ^ 



with, or can leqitimately be equalizGd by statistical ' means ^ 
\no sensible con^^arison can be made. The simirarity of; PDC ' 
and comparison groups at each site. and at all sites 'pollecA 
tively is examined in this raport as a chSck on the /premise 
of group compa.rabili ty . j / / * • 

Pr esent sample size anj P^QiG^ted attrition; . / In addition 
to the recjuirement of comparability/ it is also importrmt 
that the PDC anci comparison testing samples remain large 
enough as time ' passes to ptermit continuing analyses of .theif 
rela.tive pesrf ormance . Attrition will qccur ,in,evitably ^ and . 
the smaller ^the groups becorae, the m.ore difficult^ it* ,will . 
be to separate MC'^ s effects from the effects 'of ^the many , 
other factors that ^contribute to ^the pe^forrnance p€ the 
children involved. / ' ' ^ : - . . 

In the fall of this year. Program Year III / a check' 
v/as made on the number of* children , from the Year IT PDC 
and^ comparison JiGad Start testing samples ,who had enrolled , 
in; kindergarteTi at PDC and comparison elementary \schooIs ^ ^ 
respectively. Non-enrollment obviously , 'constitut.es ■ V 
attrition. Since last year ' S' children (Cohort^ 1), conve j ^ ' - ^ 
from the^ same populations as this yearns target children 
(Cohort 2)^ the^Cohort 1 attrition rate proyides a basis ? 
for estimati^~g"^the Cohort 2 rate. Projections of year-to- V ,^ 
year attrition from^ Cdhort 2 are presented and discussed/ 
in this report. ' _ ' ' ' 



Orqanizatibn of the Report 



dhapter II, Methods . The data ^cqI lection data . ■ 

analysis prdcedures followed d'uri'ag; the-'durr'ent/phase of. the 
Impact Study are described heref^^ These, descriptions^ document 
the origins of the data presented in , this report. 

Chapte r Hi, Findings . .This chapter presents: 

:/ . tabulations of critiadll characteristics of the \ 

in&truments in the battery, 

^.o comparisons .of the ; characteristics of *the PDC and 
comparison groups,/ both by site and. collectively,. 



I 



a:i^KM.;:un(*/i t:^ o 1^ r^amp lo s i z e and attrition rate for 
thr* groups in each site/ • . * ; ^ 




/ 

Chapter IV, CQnclusions , Findings on cjroup cdbparability , 
'adequacy the samples, and adequacy of the instruments are • . 
summarized hare, and prospects for co^ntinuation of -qhe Impact 
Study' are considered in • jfe^he light of ^^rese findings * The 
crux of the deliberations is whether oV not it will be possible 
to provide satisfactory tests of OCD'sr major hypotheses^ about 
PDG ' s effects. on children, . ^ ' 



Changes in -Plans Since th 



.Last Re porting Pey4od 

Cancellation of Teacher and-Parent Surveys , 
been noted, it is expected that •children. 



As has^ 
rents , teachers , 



and the school organization will all be affected positively 
by PDC, and .the evaluation hae been designed^expressly to ' ' . 
provide for as^sessment of^ the program's consequences in 
each of t^^se four ..domains . Large-seal^, .surveys of teachers 
artd .parjemis w^re to 'have been the prinqipal means , of measuring 
the^^ impact felt.'in these groups. Th'e lurveys originally 
■,were to hav#^ beei^\ conducted in ..8pring/r976 , ^ All^ the 
necessar/y preparations were'- mad^, but^^ plans were postponed 
because/of delaya (in the. ptocefS of clearing the forms 
with^ tl^e Office, qf Managem^ent and Budget (0MB) , The. surveys 
were rescheduled' for. spri^nq 1977,, and although 0MB apprp-val 
d't^ill not been r^ecei^t/ed the ^rst of this year^ 
iratiohs were made pnc^agaln, in anticipation' of approval. 
2ver, in; February 0MB announced that -the' survey plans 
Ld not. be approved '^t all, nor wouldvplans be approved 
a full-scale s tudy/ of procrram . implementation in PDC 
and comparison school^. The re^S&Qs /for the 
not yet been- .stated , ythus nO/accourIt of them 
here.-^ ' . ■ r1 ^ ^ I 4 




decisio-n have 
can be offered 




hat the decision, or some part 
hat alternative' plans might 
hat sufficient ,time will reniain 
urveys The next ■ Impact Study 
ar^ analysis of responses to 



^ ■ Although:, it is /possible 
of it, wili^be reversed, or 
be^ approved, if is unlikely 
in this year to conduct the 
report v/as to have contain 
""these surveys,' prc^sentefl' in^'/terms- of differences between 
PDC versus comparison' teachers/and PDC versus comparison 
parents , and in 'terms of connections discovered between 
program features and teacher/parent attitudes. Now, as 
G':^tiGal -as this Information may be to a decision about ^PDC 
f uture , i t' will^ be inaccessible, at J^east for the present 
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p^layed commencement qf fall testing in Florida ■ Last 
year, concerns . arose about the size of the; Florida PDC and 
comparison samples, and about attrition among the children 
there.' These concerns led to deliberation over the site^s 
ptatus in the evaluation. -To resolve the issue, OCD and 
High/Scope conducted a joint study on=site in October 1976. 
The conclusion of the study was that the number of children 
likely rqmain enrolled from Head Start through grade 3 " .. 
is at least high in Florida as it is in/other sites. 
Thus; it v/as decided that Elorida would remain in the evalu-^ 
ation. However, this decision was. made about one month after 
the time when preparations for fall testing ordinarily would 
have begun. Consequentry"^""testing began three weeks later 
in Florida than it did elsewhere, and the data could not be 
procesBed in 'time for this report. An account .of the results 
of testing there, will be presented in the August 1977 Impact 
Study repor^t. ' 

Addition of comparison elementary schools . ^ Since the 
last reporting period, some changes have been made in design 
nation of comparison elementary schools for three sites. 
These changes were inrtiated, jointly by the PDC coordinator 
at each site and High/Scope to ensure %hat when children s 
in the comparison Head Start testing samples progress to 
elementary school, the greatest number possible will go to 
participating comparison schools rather than to non^participating 
schoors, (Children who go to designated comparison schools 
can be tested periodically over^ the longitudinal term ^^f 
the study, ?but children vvho go to other school! may be lost 
to the study.) The. following schools "have thus been added . 
to the lists that appeared in Appendix F of Interim Report 
IV, Volume 1 : ' ^ ' 

Site , * Added Compar^i son Scl^ools 

lowa^ ■ Cassidv, Edmond, Phillips, stowe ■ 

\ (— ■ 

Michigan ' Malkifh, Rogers 

Utah Liberty, Lowell 4. . 

^ Administration of the PPLAT in Texas . In spring 1976,^ 
^the Preschool Productive Language Assessment Task (PPLAT) , 
a% Qxperimental procedure developed at High/Scope, was 
,a.dministered to a-small number of the children in the 
ConnGctrriut and Iowa tostinq samples. (The .PPLAT is designed 

. " : \- • ■ ■ ' ■ 
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to yield measures that characterize children's spontaneous 
use of language in social situations,) ' Thk resulting data^ 
were analyzed in con junctionM^ith other PDC^data that had 
already been obtained for tKe same children. The ^rpose" 
of this analysis was to explore the merits of incorporating * 
the PPLAT in the PDC battery/ ^The investigation produced 
the conclusion that the procedure does yield measures of 
language behavior^ that complement the-, other PDC measures in - 
interesting and potentially useful ways. 'For example, CGrtain 
PPLAT variables seem tqbe related to teachers" ratings of 
children ^s social demeanor, but not at all to the children's 
per formance ^'on cognitive -tests , while other variables produced 
by the same^ PPLAT procedure seem to be related to test 
performance but not to teachers' ratings. Although the ' \' , 
sample was too small and the analytic procedure- too / ^ 

exploratory to warrant formal presentation of results here, 
the PPLAT wa^ found sufficiently promising to justify its 

^ continued refinement as a tool for the PDC evaluation. 
Accordingly , \it Was -decided to continue "'idministratiQn of 
the PPLAT (with some modifications , ^made .^n light of the. 
recent findings) at a single site. 

*s . ' 

The decision to continue^ thfe PPLAT at one site rather 
than all, or rather than none, was^based upon a concern 
for 'using the evaluation ' s ' resources ^to best advantage. 
Since the test's validity is yet to be demonstrated, it /■ 

'would be premature to administer. It to all the children^ 
who. receivi the other tests, but because the : PDC /batte-ry 1 

.lacks a naturalistic measure of language proficiency and 

.because this one is promising, there is much to .lea'rn f] 
its continued application and refinement. , 

Since development of language proficiency receives 
' greatest emphasis at the PDC Bilingual/Bicul tural Demonsttation 
sites, it was deemed most practical to continue the 'use ofr 
the PPLAT in Texas, the site with the largest number of ( 
Spanish^speaking (and thus potentially bilingual) students. 
Administration of the PPLAT was begun in early Pebruary* 
at that site and will be completed by ^early March 1977, - 
■ The procedure IS conducted in each child.' s ^domi,nant language, ^ 
.either Spanish or English (as indicated by earlier PDC,_" . 
^assessments), and also in the altern.ate language, if the. 
^'^child possesses that capability..^ ' ' > ' 



Plans f oy^' the ^Next Reporting Period 

Preliminary a nalysis of PDC ''S' impact on childrgn . \By 
sprin^^ 1977, children in the PDC Head Start testing samples 
will have participated for nea-rly a full year in the PpC 

;program, .and it will be possible then to conduct meaningful 
%ests of" the hypothesis of PDC group gain o^er the comparison 

' group. The next Impact Study report, scheduled f^r August 
1977, will present the results of this first test of the . 

. hypothesis - 

Prelimi nary ana lysis\ of relationships b etween imple- 
mentation and impact . The Au'^'st report will also present i 
the results of an~analytic search for relationships between 
implementation variables and impact variables* This analysis 
will sdek to establish connections between measures of 4^sire« 
outcomes '(primarily, children's test performance) and the 
degree to which various ' aspects of ^the program have been 
realized from site to site (as determined by the Implemen-- \ 
tation Study) . 



Analysi s of gP LAT data . ^The next repott will include 
an account of the fFrst" formal analysis of data from the 
PPLAT adminirsteTed in Spanish or English or ^both to all 
children , in -the PDC a?id comparison Head Start samples in 
Texas. • , '■ ^ r-- 

v. 
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METHODS ^ 



To establish 
data of^he higbe' 
were instituted 



Data Col lection PrQcedures 

data collection routine that would yie 
possible Quality/ the following proced 



Id 

ures 



L 



anizational structure for individuals 
e'd in the data Gollection effort was out- 



1^ An org 
-invol V 
1 ined , 

detailled training manual was produced 



data Go^lection effort was 
role responsibilities w@=rfe- defined , anc 



2. A training model was designed that specified' ^ 
tester performance standards and provided ftet=a^ 
6- to 8-day ^tester training session with large-- 

' groups small-group and ■ individualized instruc- 
tion, daily r.eviews of each; tester - s performance , 
and di^^scussidn^ of potent^J. problems* 

3. On-sitie monitoring ^ of testers by\tralners was 

conducted prior to the start of the actual testing. 

j -- = ' ' - . . 

4. During the data collection period, testers were - 
resporjisible for monitoring each other's perfor- 
mance! on a weekly basis. 

5. . Site coordinators collected completed data each 

week and checked it for obvious errors or omissions 
before__sending it to the High/Scope Foundation. 

' . ' ; ■ " ■ ..^^.^^ 

Each of these procedures is discussed below., 




sts and othq^r instruments used, and the order of their 
stration, a'ra_ described, in Appendix A. Further details 
ting,^ monit^Frf^g, and other procedures follow^ed by ■ 
s on site can be found in the Field Procedures Manual - 
Scope, September. 1976)^ ' ' - ' 
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I ^ield Of ganizatiffn ^ 

The roles of the personnel who conducted f ie^d. data 
collection were explicitly defined in the Field Procedures 
Manual in order to systematise responsibilities . For 
example^ site coordinator respon^'ibilities^ included contacting 
the PDC coordinator regarding fc^fe start of testing; setting 
up and chairing a meeting with the Head Start teachers^ 
involved in the evaluation^ or contacting them individually; 
keeping i^ contact with the supervisor of field operations 
about the status of data collection and any problems ttiat 
tfie site was having; checking all completed data on a weekly,, 
basis J keeping .up-to-date ^records on the status,, of the data 
.collection F carrying out any needed training; testing (and^ 
in some cases ^ 'observing) children; and monitoring testers* 

Training Model ■" 

_ 'i' 

Two training sessions were held in early September^ 
1976-^a- -2i5-day session for the eight tester trainers and 
a '6^day (or,, in some cases^ 8^day) session for 34 testers/ 
observers i-he following weekT Since all but two of the 
eigrtt trainer^s were .experienced^ the 2-^-day trainer training 
session was more of a "refresher" course consisting of a 
review of^the child measures and practice in testl apminis- 
tration* because the decision to, include the .Flo^i^a PDC 
site in the fall 1976 data collection effort was riot made 
until October, training for the two Florida testers was 
conducted on--site in mid-^October ^nd the ^observation data 
were collected by a High/Scope staff member at. the end of 
the month, ' . 



the Eall data collection started the week of September 12 
at all sites except Florida . The length of the data/ 
collection period was fairly constant across sites^ with 
most testers finishing v/ithin nine or ten weeks. In 
Florida^, testing began in' mid.-October arid took 10 weeks * 
to complete*' Of ^th% 36 testers involved in. -the fall 1976 
testing^ ^26 were experienced PDC testers and 10 were newly 
hiredr Jn two sites^ Texas and California^ all the testers 
were new while in many of the dther^ sites an additional 
person was hired because of^the anticipated increase in 
sample s%ze. The number of testex^s pet^ site ranged from 
2 bo 4 depending on the sample size.^ 




Tester traininq. During the 8-day tester ^tra"inii{g 
session^ each test was explained and demonstrated to the 
entire group; -this was followed by small-^grouiD practice. 
The initial .small'-group practice session .invofved the use^ 
of test ^'scripts*" The scripts ponsisted of ^test instruct 
tions^ child responses, and rationales for scoring. In 
using the scripts, two testers would pair up and one^ the 
"child^" would perform as indicated on the script while the 
other tester administered the test without the script. 
This provided an excellent learning. situation* since the 
chi^ responses included on theV script covered all the 
. adm^istration rules 'and gave the testers a chance to work 
with* and correct each other without having to have a trainer 
nearby to answer all their questions. Two scripts were 
written for most of the tests. Whenever possible, new 
testers^ were * paired with returning testers so^that they 
could learn from them. 

Since i^ is critical that testers administer the tests 
in a standard way, each tester was systematically "checked 
out" on all of the child measures throughout the training 
session. During this procedure a trainer played the role 
of the child (also' recording the "child *s" responses) while 
a taster administered one or more of the child measures 
to her. Prior" to these check-outs the trainers had decided ^ 
how a trainer (acting as the child) would respond to each 
item on each tes'ffT This was done for two reasons: 1) to . 
insure that each tester was exposed to the same situations, 
and 2) to incorporate child^ responses that covered all test 
administration directions. For example, on the PIPS inter^ 
viev;, there are specific things'* for the teste^ to say if the 
child gives an^ unrelated answer, a repeated answer, refuses 
to answer, and so op. By exhibiting all these befiaviors 
■xin the check-out situation^ trainers wei^e^able to 'assess f^he* 
tester's understanding and expertise in administering Baoh 
of. the^ ^chi^ld . measures . " " . . ^ 

Standards were set for acceptable parformance during 
tl^e tester check-outs, and if these standards could, not 
l5e met, additional training and practice was prescribed* 
(MiOc'k -ou t n ^ wcM o thun repeatGd at a later time ^during the 
traininq session to insure correct test administration.. 

The eight tester trainers met every evening during 
the training session to discuss the day's activities and 
to report on the progress or status of each 'tester. Potential 
problems were identified and discussed during these meetings, 
as were necessary schedule changes. 
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Observer training i The observer draining session was 
held two days' prior' to the 'tester trtini>g session. Thus, 
the tester-Zobservers attended an 8-day session instead of 
the -regular 6-day training session.. Because It would be 
very difficult for each tester, to master both the observation 
system and the child tests in the amount of time allotted : 
*£or training/ the decision was made to haVe only one ^ person 
from each .site collect the observation data in addition to 
administering child tests. In all but two of the sites, 
an expe^Eienced tester (one .who had collected observation 
data in the past) was selected to make the classroom obser^ 
vations, .The new testers in Texas and California , however, 
had to^master b'D.th an unfamiliar observation system and ^ 
equally unfamiliar tes t ^ bat tery . - \. ■ 

The revised cLassroom observation instrument was 
introduced to the observers as a group, with trainers 
demonstrating tlie behaviors refdected on the instrument 
and the observers identifying the behaviors. After the 
PDC Observation Training Manual was reviewed and discussed, 
the observers, in groups of twos and threes, wrote scripts 
incorporating various behaviors from the observatio-n system 
categories, ^ They. then acted out the scripts while the 
other observef.s coded the behaviors • A .better under^ 
standing of the behavioral categories and items was gained 
^by this script-writing and acting. Some videotape coding 
of preschool^aged ^ children in classroom settings was also 
done, primarily to f amiliarizei^ the observers with the . - 
videotape system and the audio'" "beep" which signals 
observation intervals prior to their coding of the scetmw^^ 
on the reliability' tape , The reliability tape, which 
consisted ^Df trainers role^playing classroom scene^p^ was 
coded by all observers mid^way tlhrough the training ^ session . 



Monitoring 



On-s ite monitoring . At all sites, data collection 
activities started" the week following the training sessidn. 
Firsts the testers met with. the PDC coordinator and the 
Head' Start teachers involved and spent enough time in the 
classrooms to bGCome acquainted with the children. Also,, 
new testers practiced administering^ the tests^to one or 
more children (children' from outside the PDC and comparison 
samples) during this time. 



, During tte— t^;eek of September^ 20 a traineri visited I 
each, site for two ^ays to" monitor" individual testers as 
they actually worked^ with a child, ^IthougH it was not 
feasible for the testers to administer: the tests tcr^children 
at th^ir Midhigan ■ training session^ it was important that 
each tester have such, experience and the opportundty to... 
^receive feedback before starting the --data collection* *^ftar 
watching a tester^ the trainer presided any additional f eedr 
back 'to the tester tfiat was necessary on improving his/her 
interactions with Che children. This procedure served 
two purposes:. it gave the trainer an indication of how 
well individual testers could establish rapport and^^nteract 
with children^ a^^*€^T^lped alleviate ^ somfe of the anxieties 
the inexperienc^ff testers' felt about admi^stering the 
measures t'o chi^Mren. No. testing of PDC or comparison , , 
children was don% prior to. this^ monitoring visit, - 

Weekly monitoring « During the course of a testing 
week, testers alter^iately monitored each other; the one 
acting as monitor simultaneously completed the test booklets 
and the individual monitoring forms for each test. After 
the session, the monitor and tester discussed any errors, 
*and the , monitoring booklets and forms were sent to the . 
supervisor of field operations at the High/Scope t'oundation 
to be reviewed. The' monitoring forms can be found in ^ 
Appendix B\ The categories on the forms beside which an X s-^ 
appears are those in which testers/ as a group, made more 
errors than eKpected oxi than^^was judged tolerable. These 
areas are discussed in more detail in a later section. Ease 
of Administration p 



Weekly Pre-Transmittal Data Ch ecks ' - ' 

Testers were require^d to give or send their completed 
data to the site coordinator at the, end of each week. The 
^ite coordinator then checked, these tests, plus^ any she/he 
had completed/ for recording/scoring errors, (Site coor- 
dinators and- interviewers had reviewed a checklist specifying 
what to lopk for w_^n reviewihg each completed booklet, e*g,, 
"Is the identif icati&^^^omplete?" "Did the interviewer fail 
to give a - second trial when jt should ha^ve been given?^' ^ 
"Did the intervicjwer skip an^ item?") Errors were pointed 
out to the particular tester and/ if necessary, further' 
training was provided by the site coordinator* The site 
coordinator also kept track of all completed data (in addition 
to the individual' records each tester kept of his/her classes) 
and mailed the completed data to the High/Scope Foundation 
on a weekly basis. % - ^ ^ 



Re 



>and- Scoring of Data 




In addition to the sit© oberdinators " pre-sflbmittal 
^k, m®st data collected by inexperienced testers werev 
checked by the supervisor of \^f±eld operations at the 
^S'cDpe Foundation,^ Ei'rprs in recording or ^doding were 
idifntif ie4^ anrf^ eKplained td^^^he site coof dirrator^ >/ho therf 
discu^^d them with the othej' testers, - " * 

Once tha. raw data had been scre(^ried . for accuracy at^ ^, 
High/Scope> th^ were sent to the data proGessing section 
to be tagged witH unique identification nuitibers for each 
studfent , sco^eA and verified,* then Iceypunched and verified, 

Data Gol feet ion fe&quenGe - 

TO' facilitate data coliectionV each PDC. and comparison 
class was assigned to one oij' more of the! site 's testers 
(depending on the' number off'^classes at, % particular site) 
and the order* of classes to be compaetladv.was 'specified* 
Two factors, were taken intd^^acco^nt fn making these class 
assignments: 1) the data' Aiad't^ be; collfected simultaneQUsry 
in' the PDC and comparison schools to ^irisur^e that both groups ^ 
wer^ observed/tested during the same, tirne period,, and 2)' each - 
tester had to be assigned-' to test "both PDC and comparison 
children, ..thus eliminating the possibility of tester bias 
for either' group. \ ^ ^ 



Determining Child ' ^ Language Capabilities , 

* -.^ - 

The procedure follpwed by- testers in determining the ^; . . 
language capabilities of children in the testing sample . 
was 1) to ask the child's classroom teacher for his or her 
judgment, 2) to observe the child's v^erbal behavior in ■ . 
natural, classroom ^ cionditions , and 1) on the basis of these - 
indicatippns , to Administer the"^ English or ^^anish. ve;3rsion, 
or bo^tlj'4/of the Bilin,gual Syntax Measure (bsM) , tn most 
cases , f#iis screening ^process p^qduced consistent con^clusions , 
and subsequent .testing was accordingly cond^lptdd in English 
or* Spanish or both. ( l^n^ome cases this sgr^erting p^cess 
Led to the conclusion that a child was prof^^ient in ? some - 
third language , but not English or Spanish J These children ^ 
wereNeexcluded from the testing sample.) When the screening 
proceifes pBoved inconclusive, the tester carefully weighed 
all available information to reach a conalusion about^ the 
child's lyanguagG capabilities. . ' 
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Hindrances to ^egting , Schedule * 

- As wai the case during last yearJs data collection 
effort r there ^were/some^ ini^^ the data 

collection underway, such as iaS^;6£ permission slips or - . 
testing space . Also-, since HeadvStart enrollment had not .^.f 
been aoncluded by late September, the testq^ themselves - * 
had to make th^ir own rosters and add childfen to them as-^>^ ^ 
they were enrolled. Artd in. some sites the PDC program had 
not identif ied 4-which 'comparison He^^^ Start children would - 
be attending comparison elementary sdhopls, and- testers 
had to spend time 'looking up addresses and subsequent elem*en- 
ta'ry sdhool designations 00 that onl^, those children who ^ . ' - - 
would be going to ;the designated comparison^ ilern^ntary 
scWools would be tested* i:- * - 



' Data Analysis Procedures v 

Interim. Reports. Ill' and IV (March and August 1976) have 
'previpusly presented analysas of the adequacy of the instru= 
ment battery for .use in a longitudinal evalMtion of PDC, 
These .Analyses were based primarily upoW data* obtained for 
0L pilot" sample of children (Cohort 1) who preceded this 
yearVs target sample (Cohort 2)-, in PDC and comparison Head 
Start centers* Since it is Cohort 2 that actually will 
be followed in the pr^pective longitudinal si:.udy> the 
analyses described. in^ this report were performed to verify . 
that the earlier results hold for the present sampled 

The analyses were conducted in two ^stages , the first 
of which focused on characteristics jof the . instruments ^ the 
second on characteristics of the samples * Proceeding in 
sequential steps from the beginning of Stage 1 to the end 
of Stage 2, the analyses eKaminedi 

• reliability of the instruments , 
•■validity of ^ the instruments, 

• ^'Cross-time^ and cross-sample congruence of relia^ - 

bility and validity findings (ekamination of fall 
1976 data in relation to data from fall 1975 and 
spring 1976):, 

m factor structure of the battery, 
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• adequacy of pr^#nt sample "elies .in view of * 
projected* attrition. 
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These steps , ^Qutlined in Figure 1, are described in greater 
detail in the foUowang text * ''^Fldw chaitV) illustrating each 
of these; desGriptibfis^ appear in Appendix D.^ 

Step 1% dSqs the Internal Consistency Coefficient 
Indicate Reliability? _ • 

Cronbach'-s alpha^ ^ a measure of internal consistency , 
is an index of the amduht of overlapping variance among the 
items' thmt comprise a scale; it jnay be conceive4t- A^^*:fche m^an 
of pq^gible split^half vcortelmtions (e.g., a^'^oifelation 

of the odd and" even ffrems^ within a sc4le) , , C6e,f ficient alpha 
is a measure of reliability^ i» the sense th^t^it tefleqts the 
degree to. which the constituent "items of a test relate to a 
"Unitary construct-^the degree to which they^end to measure , 
the same thing. - - . 



^The paychometric analysis p'rocedures described in sthis section 
refer to all the instruments in th^ battery except the Preschool 
Interpersonal Problem-Solving | fest (PIPS) and ihe Classroom 
Observation System. Internal 1 consistency analysis is inappro- 
priate for the PIPS, so ^te;ps 1 and 2 were omitted from the PIPS 
^anals^is sequenCeV • The analysis of the Classroom Observatio|i 

'i^stem is treated separately in Chapter III. Psychometric 
analyses were of course preceded by scoring 'of the tests ^ and 
a commentary on the scoring of some of the tests is -presented 
in Appendix 

^Cronbach's alpha is a generalized form of the Kuder-JRichardson 
formula '20. The generalized formula isf • ^ ^ 



a — 



(1= E^i 



t 



where : 
n 



number, ^of items * ^ ' ^ ^ 

^i = item variance (after, a priori weighting) 
* — ■ 

variance of tHe scale 



V 



CfT CronbachVL- J. Coefficient alpha and the internal 
structure b-f tests . Psycho metr ika , 1951 , 16 , 297--334 , 
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Figure 1 



Mas tferr, Flow Chart for Analysis and Interpretation of Data ' . 



■■-f--.^w — ^ 

step 1 

Is the measure" rfeliable 
for thiC sample? 



Yes 





P 2 




Is the measure 'valid 


fbr 


'^his sample? 








Yes 





Step 3 . 

Are reliability and 
validity constant across 
time and samples? 



Yes 



n 



Step^ 4 

Does the factor structure 
support'^ expectations? ^ 



Yes 
— 



* . V step 5 
Are PDff^arifl comparisori 



groups comparable^ 



- Operatidbs 

Aetion decifions 
^ ^ Yes /No flows 




lliminate ^ ^ 
justify as' "necessary. 



Yes 



^ Step 6 ^ . ^ ^ ' " / 

Are\i ample si^es and 
retefitlori''^rat^B adequate? 




Consider implioatiohs for 
evaluation plan. 



Recommend that the evaJjUa- 
t ion proceed as planned 



The procedure' followed 'for the . determination of fall 
1976 iriternal consistent is pictured in Figure D^^l^ AppendiK 

/"'Dr * internal consistency was determined for each, raeasure 
lor the aggregated samples of English-dprninant and Spanish- r 
dominant children taking each test-^' If the alpha. for^a 

■i. measure wae^ greater than . ifey, within either ^language ^^mple , 
'the 'jmeasure' was considered intetfnally consistent for that " 
, g^oup; if: VHe; alpha value for , a measure fell below ,65r 
that measure Was likely /to be dropped' from the analysis 
procedure. ' ' ' " ^ ^ 



Cronbach considers alpha to be ;a /cqef £ icient of equi^ 
valence, which/ for a test Qomposed; of J/^.^latively homogeneous 
.items, approKimates parallelT^fbrnfirelimb^ility . This should 
be distiriguiihed from a coeffici3ent\^fi;'#tability, such as 
-a test^retesi Correlation over a^ititrie' interval . Cronbach 
himself does not set a criterion level for alpha (*65 hat 
selected for^ this evaluation, based upon a review of 



no cms) 



Cronbach suggests that a test whose items measure^ more / 
than one ditnension (a heterogeneous . mix of items), having a / 
low common^ factor variance and therefore a low alpha^ .might / ^ 
still have a substantial test^retest correlation, and. be 
considered stable, reliable, and interpretable , In other 
wordp, ai^ha is riot the ultimate, form bf teliability . It is, 
rather, a useful index for estimating reliability bf a homo^ 
geneous test at a single administration. 

Step 2i Arm the Measures Valid? - / . • . ^ ' 

The 'pirocediire^s %fbllowed for determining^ validity are 
pictured- in Figure p-2 , Appendix D. with' reliability , 

prior research provides sonfe information on .the validity 
of the measures , but validity also must ^b^ ascertained 
within the context of the PDG evaluation., skost' measures ' 
were selected from larger existing batteries , and itemB'on ^ 



i^he^nc 



iglish^dominant sample includ,es all English-dominaht , 
non-handicapped students for whom each measure was obtairf^d.' 
The Sptfnish-ddminant sample includes all non^handicapped , 
Spanish-dQminant students for whom each- mWsure wa-^ obtained 
ih Texas and CalifDrnia. The ^^rocedure determining 
lanquaqo flom.inanco is dos.crU:^bc3d above under Data Collection 
Procedures"- , = 



It . . * 
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most-df^ the measures ^have been modified^ bbth-to meet the 
ripeW of the .sample beihg^ tested and to p'frnvit use by para-- 
prof essipnaY^testers . Therefore ^^validity of the niea suras 
within the RDC environment^ and within the teat battery^ irf ^ 
which they are administered ^ must be determined anew. The' 
concern in this report Is with cbncurrent validity-'-a 
measure's relationship to other measures of the same con;?i^; 
struct and to measures of other ^constructs; p measure siibtfld ^ 
correlate highly with othewi;meW the saijie coristructs, 

and, should not correlate at ali'^ith measures independent 
cons,truc,t& . 

*\^*'An hypothesized correlation matrix was constrilcted 
prior to the fall 1975 data analysis, based on knowledge of 
the constructs the jneasuK^es^ were presumed to represent , ^ 
The ^^Mliies in tftf rtfatrix indlcat§ the level of relationship 
that thepretically should obtain between any two measures if 
they both 'genuinsiiy ,mea^^ the constructs they a^re supposed- 
to represerit* The ictual fali 1976 correlations (within - " ^ 
language groups) were then evaluated against the hypo'thesiEed 
correlations/^ ^ , ^ ^ 

The hypothesized correlation matrix was constructed ' 
by. first determining the correlation within^the three 
areas of child tests: ■v cognitive^language ^ psychomotor f_ 
ind social-emotiqnal measures, ^ Then the desired corre^ 
lations among the three groups of testis were .determined ^ 
Generally^ highe^-^correlations wete^ expected wi^thin an ^ ? 
area than between areas* But since each. area' is actualrly^ 
composed df^i^ikea cdnstructs rather than aTite . na i ive , 
measures mfi th^ same construct^ very few high :or^ alations 
were . exp^^ted / , , / , . % 

¥ - ' ^ ■ A ' ' 

^ ' The fall 1976 correlations between pRasures ^( the ones ^\ ^ 
^ found relifble) were calttulated^ within .each^ language groups 
%nd the following p5,oced>jire was used tb determine whether ^ 
■a given mea'Bur.e was valid* First, the oyFained ^inter^ 
correlation: matrix w.as compared with the hypothesized 
matrix and deviations of each^correlat ion from the hypothe- 
sized one wera calculated (e.g*, if .the hypothesized 
correlation^ was^, "medium" and^that obtained was "low," a 
deviation "-4" was scoredr if th^e , hypothesized corre- 
lation wa^% "zero" and that obtained was "medium," a 
d^viatiorl^jpf " + 2" was scored). \ For each measure, the 
absolute -^Mues of the deviations were summed across, all 
measure^' and divided by the number; of measures. If this 



ratio h^<3 a, valu^of 1.0 6r less, the measure was considered* 
.valid- -The cry^'ion implicit in this procedure is th^atf 
a measute ' s cqf^TOurrent validity is adequate if, on the^ 
averafe*) the obtained ^cprrjlations with others measurestara 
within ttte range adjacartt' to^ the^ expected value. ^ ^ 

^ t^'-^ ■ ' 7' ^' ■ ^ ' _ ^- _ 

Although this procedure- allows for rather large deviations 

from the hypotheeij:ed relations, it still provides a useful 

f'frst' approximation ^to validation of the measures. 

Step 3 : Are Rellabiiity and Validity Constant Across 
^ime and Samples ? 

This step, illustrated in Fi,gQre/D-3, Appendix D^ i 
similar to the step described' in tfie last report fpr .as0__ 
taining the f all=to=spring constancy of reliability ^nd 
validity estima%€s. Basicttliy, the internal ''corisisteney- 
coefficients ajid validity Estimates obtained -f®r each measure 
for each of the sampled were ^^ompa red and a ^^rite^oi> of 
constanQy was applied to assess the stability of the figures 
across time (fall^l975, spring 1976^ fall iL976) and across 
samples (Cohort 1, Cohort 2) t \ ' 

The data. analysis procedures outlined in this chapter 
and in previous reports Have been used to determine the 
psyGhometric characteristics of the child measurement 

' Is^aitery for the samples actually tested in 1975^76 ('Cohort 
1) arid 1976-77 (Cohort 2), These analyses also^ have a , 

^broader goal i V detarmining the^ siiltabilityi..of . this battery ' 
for any potential, t^rg,et populations of childreri that PDC • 
might serve in the future Since data^are^now available ^ 
oh twosf separate samples from this larger ^population , 
cotTiparison. oF the reliability and validity estimates may 
suggest, more generally the appropriateness of the measures 
as todlp for this type of evaluation . ThereforSe relia-- 
bility coefficients and validity estimates from the three 
available time--points were eKamined with reference to the 
question of their constancy. In addition to assessing - - 
the aonstancy of reliability and validity estimates, these 
analyses may indicate the appropriateness of eventually 
equating the English and Spanish versions of these measures 
in order to relate scores on tests, given earlier in one 
language to scores on the same tests given later ip the 

i^alternate language. ^ .. . ^ 



step 4; Does the Faetor Stmctupe . Supp^ / ^ 

For both. English^ and Spanish^dominant samples, a 
principal cqmppnents factor analysis was performed on all V 
scale: acoree "found to be reliable for this time-point. 
After, varimax^ rotation, a table q£ factor loadings for 
each ^language ^ sample (English^dominaht, Spanish--dominant) 
was Constituted. Factors were named or described on the --"^ ^ 
basis of the constructs feprep^ted h^ . the tests having thi. 
highest loadings on each one, ' Comparabi-lity of theig .fall 
1976 results to previous analytic results' (fall 1975^ - 
spring 1976V was examlmfd and cross-language-sample comparisons 
were made. The struct^© of the battery and the stability 
of that structure were" thus ^determined*; aiid implications 
were^'drawn for the: future. ' ■ 



Step 5: Are PDC a^nd Comparison GroupB OompArablel ^ / . ^ 

PDC and comparison groups were first compared within= # , 
.each site on all the scores produced by "the test ^)atter^ and 
on variables representing backgEpuhd *characteristip^ ^ ' (l£ 
data were missing on any background variable fo:^. merffevthan ' 
25% bf ihfe "^hildten in either gro^^p, that var^ f-iis 
omitted from thS\ analyses %or^ those %^ups, f'' For categorical 
variables . (e,g, , ethnicity, sex) , equality^ of PBC. vs, . 
'compatison group proportions was e^valiiated by means of ^X" ' ' 
analyses ; the^ criterion for con^cluding that^a group difference 
existed^ was a, statistical significance value of less than 
.01 (p<,01') Next^* all groups were combined als toss sites ^ 
rtito PDG and comparison aggregates and tlie same * anaLVses 
wer^" reg^eatefi. . : ^ . * 

■ . " * " ' . ' ■ ^1 

The results of the within^site comparisons generalize 
to the prospect of cbndycting site-by--site analyses of . 
program impact'^ in Jtbe future; the results of Comparisons of 
the^ aggregated groups generalize to the prosppct of conducting, 
cross-'site analyses of impact. interpretations of the results 
address the basic' questions Are FDC and comparisdn groups, 
taken^ locally or nationally,, similar enough- to begin with 
that differences discovered in.^lie future can.be attributed 
confidently to program impact? , ' . 



t Georgia , .which has no comparison group, was omitted from h 
the w^thin^site analyses described, but not from the 
analyses of aggregated groups. 



step 6 : Are Sample Sizes and, RetentidTi Rates Adequate ? 




In order to remain in the testing sample from point to^ 
point of a longitudinal study, children' from tlie ©ci^inalr" 
PDC and comparison Head Start grDups must be .^o^efcted' in a 
designated PClC or comparison elemei^tary school, respectively, 
in each^.successive year of the study. This is .necessary for 
two reasons: first, the* /logistics of testing require a 
central testing location (or hb more than a few locations) 
so that administrative compleKities^re minimized; second, 
the Impact Study is basicaily conceived as a Comparative 
-Study of Jtwo treat^ents-'-PDC ai^d non-PDC^ — and only if ■ ^ 
children'^ remain .in a feingle treatment dan their performance 
be taken as a ref^^ection of that treatment, 

A check was made this fall on the number of children 
from the 1975^76 test ing. sample (Cohprt 1) who, for the 
1976-'77 school year, were enfolled^, iK, kindergarten at an 
appropriate school.. : Since these "children presumably come 
from , the ^ $ame populations as this year's^ target'' sample 
1 Cohort i) , the fi^e^ Start=to-kindergarten retention rate ; 
obser^fed for the earlier sample shouM provide a sound^ 
basis ^for projecting Cohort 2 retention. However, thd^ ^ 
retent4on^ rate; for the first interval (Head Start to K) will 
not necessarily be the same -as the ^ateffor latef . intervals 
(K .to 1, 1 to 2/ 2^ to 3) • In fact, it generally happens, / 
that' trie greatest loss occurs, in the transition from Head " 
Start to kindergarten. Thus 'the, figure Representing Head 
Start--to=kindergarteri re^tention cannot simply be multiplied 
by itself to project retention in later years. ^ iristead,^ 
changes in the rate of retention must be taken into afccount. 
for the Cohort 1 retention figures, this was done in the 
way described in the next paragraph. 

Last year and the'te^^' before, each site waa asked to 
provide figures from previous years that documented the 
number of children retained each year in the progression from 
Head Start through grade 3. Using these figures, a curve 
was drawn that described the mean observed rate of retention, 
across all sites, for each grade interval from Head Start^ 
throuqh qratie 3. ^ 'rhis curve, shown in Figure 2, expresses 
retontion rate as the proportion of children from an original 
Head|^Start group who can be expected to remain in a designated 
program (i,e,, PDC comparison) through each suocepsive 
grade, (fHe attritib^/ curve also shown, in Figure 2 is of 
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Figure 2 



Estimkted. Retention/Attrition C Head Start through 

* Grade 3^ for Head Start Populatioffs at PDC SiteS/ 
Based' orf Data for Groups that Pr^etSeded Cohorts 1 and 2 
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course the mirror image of the retention curve: % attrition - 
100 ^^ retention.) To project future retention of the 
/Cohort 1 children now enrolled in kindergarten^ the retention i 
figure obtained for the first interval' (Head Start to K) 
was weighted in such a way as to incorporate the general 
trend observed for. the earlier year groupq. Thus no matter 
what the retention figure for any Cohort 1 group in the 
Head Start-to=kindergarten interval^ the projeations for 
K to 1, 1 to 2^ and 2 to 3 deseribe a curve that parallele 
the curve for the population. '''(So, for example ^ if any ' 
grbup'e firsts-year reteation figure was the same as that ^ 
for *the general, population , -^the curve that is eKtended from 
that figure wlli 'qq4nc;ide exactly with the general curve.) ^ 

These productions provide the basis for forecasting the * 
size* of the Cohort 2 PDC and comparison samples in later 
years of the contemplated longitudinal study. For this 
step, a table was constructed that began;with the actual 
^umbers of childre'n included in this year ' s PDC and comparison 
A:esting samples. Next, for each Cohort 2 group the number 
was multipli^ed by .the ^^uccessive, retention coefficients 
obtained for the appropriate Cohort I'^group, adjusted as 
4escribed, This yielded estimates of the. actual number of 
children likely to remain insPDC and comparison groups, 
locally and nationally, for each prospective year of the 
evaluation". The implications of these .figures for future 
statistical analyses of PDC's effects are discussed in the 
Findings and Conclusions' chapter of this report* 




" FINDINGS 

Charalpterietlcs of tha Samples 

General De script ion 

In fall 1976, l/2l9 children were tested at 12 PDC ^site> 
Table 1 shows the number of children who came 'from PDC and' 
comparison groups at each site, and deecribes the makeup at, 
each group in terms of the handicap status, ffCAnicity, BrnK, and 
dominant language of its menUDers. ; ^ 

Not^all these children entered into the psychometric 
^anaLyses presented In this section^ but the right-hand column 
in T^ble 1 showd the number that did. Childre^n whose dominant 
language was other than English were excluded^^^f rom the analytic 
sample, except in Calif ornia and. Texas , where Spanish--dominant 
children comprised a sample of their own; also excluded were 
children* with handicaps that were judged likely to impair test 
performance unduly. Handicap information came principally from 
local Head Start records , Table 2 gives the €requency of each 
type of handicap reported among children in the full testing \ 
sample. Note that not all o£ these handicaps were judged to, * 
be sufficiently debilitating to warrant a child ' s^ exclusion 
from the analytic sample. It is possible that those who have 
beep excluded at this time will be inclitded in later analyses/ 
but for present purposes it is^ preferab]/fe to restrict the saT 
to those for whom the measures are likely to possess greatest 



In , the ^thirteenth site , Arizona / impact was assessed by case 
study techniques rather than by testing, since testihg was 
found to be unsuitable for the Nava jo-speaking chiidren there p 




Table 1 

Cornposition of the Samples^ Fall 1976 
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IOWA 
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54 
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^ Comp 


44 
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Table 2 

s. 

Types and Frequenaies of Handicaps 

Reported Among ehildren in the ^ 
FullTesting S^mplm (Total N ^:1133r 



Type 



Fr/equfincy 



(% of Total) 



Physical impairment 
HearlHfl Impairment 
speech Impairment 
Visual impai rmei;it • • 
Embtional disorder 
Learning disabnity 
Chronic illness 

er, non-deb'1 1 i tating^ 



M 
6 
63 
12 
, 1 
6 
8 
2 



.(1.1) 
(0.6) 
(5,6) 
(1.1) 

(a.i) 

. (0.5) 
(0. 7) 
' (9.7) 



Al 1 types combined 



110 



(B,7) 



Note. Children with multiple handicaps appear raultiply in this tab!, 

^This table excludes data from the Florida "sample 5.^which were not ° 
available at the time of compilation. 

hcindiccip of this type alone was not judged sufflcientTy severe to 
wdrjant a child's exclusion from the testing sample. 
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" Cpmparability of PDC and Comparison Groups - . 

Onc,e the final analytic samples had .been established^ 
analyses were performed to determine just how comparable the 
PDC and comparison groups really are 'at each site and in 
'aggregation* Both background characteristics and ^ test 
J performance w^re-axamined and the results are shown in Table 3. 
The background variables examined represent characteristics that 
have been found iji past research to be relaied to school. ; 
/ performance* ^ (If the groups are not initially comparable on 
these dimensions, it is possible that the effects produced by 
PDC .will be masked by extraneous differences unless these, 
differences are somehow taken into account.) 

i For each site and for each variable appearing in Table 3, 

the .assumption of PDC-compar ison group equality was' tested 
statistically (using the chi=square technique for categorical 
variables and t tests for metric variables) . All available 
data entered into each analysis , meaning that even if data 
were missing for a particular child on one or more" variables # 
data obtained for that child on other variables did enter into 
the respective analyses. A diffei^ence was declared to exist 

' between PDC and comparison groups if analysis indicated the 
chance probability Of the observed difference to be less than 
one in 100 (p < .01) , I , ^ " 

. Site - level £ i-ndings . .The asterisks In Table 3 mark 
statistically significant differences between PDC and comparison 
groups, the checks mark instances where the groups can be 
considered comparabl-e or identical. . 

At the individual site level the groups appear very 
similar; there are differences on background variables in only 
r one site. On performance measures ^ of the 13 comparisons made 
(the Spanish^^^ English-dominant samples in California and 
Texas were td's^d separately in. these analyses)^ ten showed = 
either no , group differences or differences only 'on the POCL 
or Height and Weight; only two sites had group differences on 
more than one child measure. For details on the natiire of each 
statistically significant difference found at the site level/ 
see Appendix E, 

Aggreg ate-level fi ndings* For this analysis , data were 
^- pooled for EngTish- and Spanish--dominant c^i-ldren separately 
by treatment groups resulting in four groups: PDC-^English/ 
' Cornparia©rt-English r PDC-^Spanish and Comparison-Spanish • - 



Statistics are also shown in Table 3 for an ag^jregation that 
excludes Georgia, since Georgia has no, con temporaneous » comparison 
group to balance its PDC group in the aggregate. Excision of 
Georgia does not alter the results, however. The combined PDC 
groups and' combined comparison groups appear to be in parallel 
with or without the Georgia PDC data. 



Tabl,e 3, . , 

Comparability of PDC and Comparison Groups at Each site 
and of Groups Agcrregated Across Sites, Fall 1976 
' ' (p < ;oi) ■ . . 



Key 1 

* ^Statistically sigftificant 
group difference (p<,01) 

/ - no significant difference 
between groups 

? ^ data insufficient for 

analysis ^ , 

- ^ test not appropriate 

- - * 


rCALIFORNI A-Engl i shJ, ■ 




SITE 




EN'GL ISH .AGGREGATE 


SPANISH AGGREGATE 


^ &■ 
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3d P 
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a 
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8 
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a- 
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H 

& 

1 


1 

in 
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1 
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N for analytic sample 


PDC 


37 


7 


51 


37 


43 


45 


49 


32 


58 


26 


38 


-61 


49 




530 


45 


487 


Comp. 


24 


15 


30 


55 




39 


51 


45 


58 


19 


34 


55 


66 


29 


471 


49 


471 


BACKGROUND CHARACTERISTICS 

Ethnicity 
Sex 
' Age ^ 

Prior Preschool EKperience 
Number of Siblings 
Mother * s Education 


/ 
/ 

/ 
/ 

? 
'p 


/ 
/ 
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/■ 
/ 


/ 
/ 
/ 

/ 

V 


m 

'H 
U) 

m 
o 

0 
m 

N 

u 

0 

a 


/ 

V 

/ 
/ 
/ 
/ 


/ 
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/ 
/ 
/ 
/ 
/ 
/ 


/ 
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^' 
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/ 
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/ 
/ 
/ 


,/ 

/ 
/ 
/ 

/ 
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7 
7 
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7 
7 
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7 

7 
7 
7. 

/ 


7 

7 

7 

* 

7 
■/ 


7 
7 
7 

7 
7 
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TEST PERFORMANCE 

CognitiVe-^Languaqe Measures 
BSM^English 
BSM-Spanish 

Block^ Design (WPPSI) ' 
Verbal Fluency 
C^erbal Mamory^l 
Ve^al Memory- 3 
Draw=A-Child 

Psychomotor Measure 
Arm Coordination . 

Sociai^Emotional Measures' - 
PIPf- Solutions / . 
Vnt'i )tal . 

pncL=2 

Health-Nutrition Measures 
Height 
Weight 


/ 

/ 

/ 
/ 
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/ 

/ ; 
/ 
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/ 

/ 

/ 

* 
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/ 
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/ 
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/ 
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V 

/ 

/ 

/ 
/ 

/ 

/ 
/ 


V 
/ 
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/ 
/ 
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V 
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/ 

V 
/ 
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/ 

7 ' 
/ 


7 

/ 
/ 
/ 

y 

V 

./ 

/. 
/ 
/ 

/ 


/ : 
7 

V.- 
/. 
/ 
/ 

/ 

/f 
/ 
w 
7 

/- 
7' 


7 

7 
7 
7 

■ > 

7 

V 
7: 

1- 

V ■ 
7 


7. 

* 

7 
7 

7 

7" 
7 . 

y 
1^ 


-? 
7 
7 
7 
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7 
7 

7 

7 - 
■7^ 

7 


7 

7 
7 

* 

7 ■ 
/ 

■7 

7 

'7 
7 
-7 

^! 

m 
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Figures' 3 and 4 show the relative fetanding.of the PDC and 
comparison aggregates on certain background variables and 
Figures 5 and 6 show^ their relative standing on 13 performance 
measures plus three more background ^ variables fts at the site 
level, the similarities of the aggregated ' groups are more 
prominent than their differences* In the English--dominant ■ 
sample, there are, no significant group differences on- the 
background'' variables and only one difference^ In test performance. 
In tf^^e ^ Spanish^dominant sample, the group^ differed on *only one 
background variable^ and tHere ' was no/dif ference on. ^ny vof the. . . 
performance measures. " , , 

' ' - ^ ^ . ^ ^ . 

Aricilysis /of '^ttfitioh Trends \ . ' . V=^' ' " ... 

' \ The first column .of Table 4 "shows, for. each 'site and fox 
all sites collectively, the number of chi Idren 'tliat^ were"^^ 
available for fall 1976 testing .in PDC and comparison Head Start 
centners* These children constitute the full sample of Cohort 2, 
the cohort whose px-ogress will be followed through grade ^3 if 
t-he PDC evaluation is extended longitudinaTly ^ On the average, 
these grotips are about 9% , smaller than site staff had estimated 
they wpuld be (the estimates provided last year by PDC boordinato^ 
can be found in Interim Report ^V, Volume 1 ^ , August 1976 ) • 
Moreover, the mean retention rate determined' this fall for^-"*'^-;* 
Cohort 1 "Children (Cohort 2*s pilot-year predecessors^ now in ^ ^ 
kindergarten) is lower than was anticipated* To restate these 
findings on sample size and attrition i Cojiort 2 is smaller than 
it was ^expected ^ to b?, and since it is likely to follow Cohort I's 
attrit iok 'pattern , its ^z^' is likely-'to diminish faster than 
was antteipated . Theae findings 'are' expanded and discussed in 
the remainder of this section* " . ' " ... * / 

/The data that had been gathered earl-ier on past retention 
ra.tea at each site 'showed that, on" the average, 68% of the 
f our-^yearvold childreh enrolled in a given Head Start center 
went on to» enroll in kindergarten at the expected scliool the 
following year,/' Among ^'CtfMort . 1 children, howQ-ver, 'the'riead .Start^, 
to^kiridergarten retention J, figure is considerably lower^61%. 
Jkjid because ' thi^s is the figure that provides the basis for 
^proj^ecting the^ number .6f Cohort 2 children likely to remain 
in future years, thofee pro j ected mumbers , too, are smaller than 
earlier projections. , ' 

J i /tt is important, to note that the number of PDC and^^omparison 
chtidre^n '^yho_ v/ilK for testing 1^^ the future 

is'likeiy to be pv.en lower -th^n thes^. pro jgjc tidns , for a nuinber' 
of reasons* First,' the figures^' shown in Table 4 represent 
children in the full samples , Although all these children cou-ld 
be tested, consideration of handicap and 'language factors , would 
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Background Characteristics of ^ Aggregated Analytic Samples 
of English-Dominant PDC ^nd'' Comparisdn Children 



PDC (N ^ 530) 
eomparison (n - 47J 



Significant 

Group 
Difference. 
C^<..&1)? 



ETHNICITy 



Black 



Himpdnic 



Native Alaskan/ 
American ^^^dian 

White 



Asian/Pacific 
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(41%) 
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SEX. 



Male 
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Figure 4 



Backgrbund Characteristics of Aggregated Analytic " Samples 
of Spanish-Dominarit PDC and Comparison Children 
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Figure 5 



Moan Staajiard Scores of Aggragatid kalytic Samples of English-Dominant. PDC 
and Comparisqn Children on Selected Demographic and Perforwanc'e Measures 



P s Mito of pic children (N = 530,) - 
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1 s Hiqh/l.ow score in oombine^roiips 



■ ! „,ii 

Nunigr of Siblings ^ 

Mother's Education j-i. 

. ' [ (-5.9) 
;BS!f English 

WPPSI Block Disign , 

Verbal Fluency 

^ Varbal Mariory-l 

Vecbal Heinory-3 

Arm Coordination 



PIPS 



^.P,= ,001 



POCL-Total 
\ 

POCL-1 
P0eL"2 ' 
Haight 
Weight ' 



■6.11 
-3^ 



C »'=<00l t 



\ P ^ .01 



C s -.01 .+ 
P 5 -.02 + 



t C = ,02 



4 



p =.-.ooH 



t C = .007 



'.02 



P s -,05 ^ ^ 



t Cp .03 



+ C B .06 " 



.08 + 



iC = .09^ 
I" 



t C = .002 



1^ 



P * -.03 + 



f C = ,Q '4 



P = -.04 \ 



H = ,05 



J 5 -,05 4 



/ 



t C ^ .06 



P -.02 + 



+ C = .02 



P = -.02 ^ 



+ C s ,02 



.02 



H * .02 



P - -.009 + 



+ C « .01 

.02 f '■ / I 



Significant 
Group Difference 
(p<.01)? 



(5.9) 



(6.2) 



(4.5) 



ERlCa 



^ + C s .02 ^ , ' 

I 1 ' ' I- ■ , I 'fe ' I %f \ 

-2 / -1 "l 0 ■ +1'' ■ +2 +3 i ' +4 

f,. 

' 2-Score Units (Based on Di|tributions of Cornbihad Groups) 



(6,2) 

I (5,7) 
+5 . 



? 5 , 



m 

NO 
NO 
NO 
NO 
NO 

.YES 
NO 
NO 
N(J 

'NO 
NO'f 
NO' 

JO. 
NO 




• , Figure 6 ' ' ' . ' ' ' . 

ard Scores of Aggregated Analytic Samples of SBanish-Dominint PDC 
arison Children on Selected' Der[iographic\,and- Perfbrprice Measures 
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. 'Table 4 • ' , 

\, , Projected R-etention of Cohort 2 Children for Each . 
" Year of the Prospective' Longitudinal Study 







1976-77 


1977-78 


1 978. 


.79 


1979-80 


1 








Head 
• % 


Start 

" N 


% 


K 

N 


■ 1 

% 


N 


1 


2 

N 


3 

% 


n\ 


California 


PDC 
Comp 


100' 
100 


44 
40 


70 
42 


31 
17 


5.9 

35 


26 
14 


54 

32 


24 
It 


44 

26' 


19 
10 


Cgjjirado 

) : 


PDC 
Comp 


1 00 

1 WW 

100 


58 
. 32 


49 
42 


27 
13 


41 

35 


S3 
11 


38 
32 


21 
10 


31: 

. 26 


17 

a 


Connecticut 


PDC 
Comp 


100 
100 




50 
^42 


28 
24 


42 

35 


S4 
20 


39 


22 

■ 


r 


IB 
IS , 


Florida^ ^ 

r 


PDC 
Comp 


100 
100 


47 
^ 39 


31 
65 


IS 
85 


CO 

55 


12 
21 . 


24 

50 


11 

■ 20 


' 41 


9 
16 


Georgia 


PDC 


1 00 


46 


77 


35 


65 


30 


by 


87 


49 


23- 


Iowa 


PDC 
Comp" 


100 
100 


50 
' 54 


51 

38 


se 

21 


43 

32 


22 
17 


39 
29 


80 
16 


\ 32 
\ 24 


16 
13 


Maryland 




100 


44 


73 


32 


61 


29 


56 


25 


|46 


20 


Comp 


100 


58 


77 


45 


65 


38 


59 


34 


/ 49 


28 


Michigan t 


PDC 
Comp 


100 
■ 100 


66 

^ 64 ^ 


86' 
'69 


' 67 - 
44 


72 
58 


48 
37 


66 

53 


44 

34 


54 
43 


36 

28 - 


Texas 


PDC 
Comp 


1 00 
100 


64 
57 


84 

76 


54 
' 43 


71 

64 . 


46 
36 


65 
59 


42 
34 


53 
s48 


34 
27 


Utah 


PDC 
Comp 


100 
100 


68 
61 


62 
36 


42 
22 


52 
30 


35 
18 


48 

28 


33 ' 
17 


39 
23 


27 
14 


Washington 


PDC 
Comp 


100 
100, 


58 
76 


62 
70 


• 36 
53 


52 
59 


30 
46 


48 
54 


28 
■ 41 


' 39 
44 


23 
33 


' West Vi rginia 
i 


PDC 
Comp 


100 
100 


46 


55 
73 


25 
■2 7 


46 

61 


21 
23 


42 

56 


■ 29 
- 21 


35 
46 


IB . 
17 ■ 


AGGREGATE 


PDC 
Comp 


100 
100 


044^ 
675 


63 
58 


408 
334 


54 3 

49 2 


45 
80^ 


49 
45 


316 
268 


40 258 
36' 809 



NOTE; represents proportion of original group r^aining^ represents 

size of group remaining* In the 1976-77 column, ^ - original sample 
size and"^ ^ ^TOO, necessarily, The^ figures in successive columns are 
'projections based on the actual 1976-77 figures. 



ERIC 



require the elimination^ of some children from the analytic 
sample, which is the source o^f the data used for statistical 
anarysis. Of the 1,219 children in the full Cohort 2 testing 

^ sample^ 124—about 10%;--were excluded from the analytic^ sample - 
for reasons of handicap or language* And of the remaining 1,09=5^^ 
92 comprise the Spanish^dominant analytic sample leaving 1^001 
children (82% of the original 1,219) in the sample that provides 
the basis for most analyses*. Thus, when examining the year'-to-- 

' year sample size proj.ections shown in Table 4, it should be 
kept in mind that, 11^ the aggregate, only about 82% of the 
children who are still present at any point in -the future will 
enter int'o analyses base^ on English language test data. 

There r^ay^ however be some compensations. Some handicaps 
may be overcome k bringing s®me of the children who are now 
.excluded beQausejof handicap's back into the analytic sample, . 
^rld some of t\he children who are now ^anish-^Ho^nant may later 
be.c^me members of the Engli sh--dominarit sample. But such increments 
'are 'likely to be offset by other decrements, primarily due to 
^^.^sence from school at testing time and refusal to cooperate^^ 
^refusal on the part of the child, the child*s parents, or school 
officials* (Of the Cohort 1 children tested in.-f all 1975 , 
3% ^could not be. tested the following spring for reasons other 
than departure from school, and "these reasons^ included absence 
and\ refusal , ) 

Taking these factors into consideration, the number of PDC 
^1)0 comparison ahildren from Cohort42 who are likely to remain 

the analytic sample of English-dominant children through grade 3 
can be estimated a^about 383 (212 EDC children, 171 comparison 
ohildren) . Estimated similarly, the number of children likely 
to remain for that period ^ih the Spanish-dominant sample is 
about 34 (l8 PDC, 16 comparison children). What "Ihese projections 
imply for the longitudinal study of PDC ' s impact is discussed 
in^ the Conclusions chapter. 

A note on the afttrition statis tic^ reported for individual 
site si The proyections shown in Table ^4 caa- be expected" to' be 
more accurate for all sites aggregated than for individual sites* 
This is because the factors affecting retention. within sites may 
not operate consistently frorn^ year to year. For example, a change 
^ in scHctq.! attendance boundaries could mean that a large proportion 
'^^Df ^a kiirdergarten^^it^r ^PDC sample might not return to the 'same 
school for first gr^&^e. Yet since boundaries are unlikely to be 
changed every year, retention of the rest of that sample from 
grade 1 to grade 2 would probably be proportionally greater. 




The Florida PDC site was affected by such an occurrence 
last year* A large fraction of the children from the. Cohort 
1 PDC Head Start group were assign^ unexpectedly to a non-PDC 
elementary school for their kindergarten year because of 
over-enrollment in f he PDC schoo 1 - Arrangement s have been made 
at that site to ensure^ that this will not occur for Cohort ,2 
children. But it is probable that such events will oqcur at 
other sites in coming years (in Iowa's^ site, for eKanj^le^ a 
busing plan is under consideration that would alter present 
school attendance patterns, and this might aff feet retention of 
Cohort'2 children at that site). It issassumed that these 
OGcurrences) will av^age out across sites^ making the aggregate 
projections in Table 4 more dependable than the projections for 
individual sites* 



Characteristics of the^ 
Indivi dually Administered Instruments 



Reliability for Cohort 2 in Fall 1976 



'Table 5 summarises findings on the reliability of all but 
two of the instruments included in the fall battery. (The 
Preschool interpersdnal Problem-Sqiving Test is not inclUded 
in this table becaus^V its ^rfoting is not ameriable to, computation 
of alpha ^ and the ^fejAability of the 'PDC Classroom Observation 
System -is treated elsewhere in this chapter*) The coefficients 
of internal consistency (Cronbach's alpha) were (pomputed separately 
for the respective analytic samples of Ehglish-dominant and 
Spanish-dominant children ^ . aggregated across sites* As Table 5 ^ 
shows, all the scales in the English and Spanish test batteries 
^meet the preset reliability criterion==an alpha coefficient of 
• 65 or higher . ■ 



Constancy of Reliability Across Time and Cohorts 

Table 6 presents comparisons of reliability coefficients 
(Cronbach ' s alpha) for the two^cohort/ three-time^point data 
currently available. An inspection of this table suggests an 
ordering of the measures rhto three classes: those with fairly^ 
constant reliability coefficients^ those that show increasing 
rteliability across time^pointS/ and those that show decreasing 
x^liability across tiiT\e^points . 



Block Dm 
Verbal 



Measures that s how cons tant r^llability . The BSM-'Engl i sh ^ 
istered in the' fall only to both cohorts) ^ 
bal Memory- 3 , and POCL-Total appear quite 
and across cohorts for both the English-- and 



sign 



Fluency , 
stable abroWs tiipe 



Spanish-dominant sampler 



Table 5 

Reiiability of the Child Measures;* 
ronbach ' s Alpha (Internal Consistency)" 
for Fall 1976 Head Start Children 



■lea su res 



COGNITIVEitLANGUAGE 

Bi 1 ingual ^Syntax, Measure -Engl 1 sh 
Bilingual Sysi^Ax Me§('su)<e-Spam' sir 
Blocksriesign (WPPSI) 
Verbal Vluency' (MSCA) 
Verbal Mftmory-1 (MSCA) 
Verbal .Memo^y-3 (MSCA) 
'Draw-A-Child (MSCA) 



PSYCHOMOTOR. 

Arr# Coordination (MSCA) 

SOCIAL-EMOTIONAL 

POCL-Total (High/Scope) 
POCL-1 (High/Scope) 
PDCL-2 (H1gh/Scope) 



Cronbach's Alpha 



Engl ish-Dominant Spanish-Dominant 
Children Children 



1001 
1001 
1001 



997 


.84 


39 ■ 


.53 


16 


. 95 


89 


.86 


999 


.77 


94 


.82 


975 


. 76 


92 


' .81 


997 


.35 


94 


,89 


989 


. 82 


93 


.84 


978 


. 84 


92 


' .78 


976 


. 65 


89 , 





95 

96 
90 



94 
94 
94 




97 

,96 
96 



Jm instruments are not includad: .the scoring &f"the Preschool Inter- 
jpersonal problOT Solving Test does not 'lend itself t| computing alpha, 
^ and^the reliability of the Classroom Observation %y%Wm was deternilned 
/ differently. 



Texas and Californi^a only (Bil ingual/Bicul tura| Demonstration Sites), 



1 



Table 6 



Comparisons of Reliability (Internal Consistency ). 
\ ^for Cohort 1 (Tested Fall 1975 and Spring 197qj 
- ' and Cohort 2 (Tested Fall 1976) 



Measures 



Bilingual Syntax Measura-English 
Bilingual Syntax Measure^Spanish 
Block Design 
Verbal Fluency 

Verbal Memory-l* ' \ 

\ 

Verbal Mamory" 3 
Arm Coordination^* 
Draw^A-Chlld 
POCL^Total 



Cronbach's Alpha 



English^Dominant 

Sample'' 

Fall Spring Fall 
1975 1976 1976 



.82 .88 .84 

(691)'^ (430) (99?) 



.88 
(IZ) 

.75 

(724) 



~ ^ 95 

(16) 

.78 .77 

(80y^ (999) 



.75 .74 ,76 

(726) (458) (975) 

.64 .73 ,85 

(724) (435) (997) 



.83 



,82 



.(72br (434) (989) 

.54 .62 .65 

(738) _(457 ) (976) 

,82 .74 .84 

^^(737) (456) ^^978) 

.90 .93 \p- 

(719) (468) (1001) 



'Spanish=Dominant 
Sample 



Fall 
1975 


Spring 
1976' 


Fall^ 
1976 


.93 

h7) 


93 
(10) 


.93 

(39) 


• .96 ' 
(85) 


.76 

(70) 


.86 
(89) 


.80 

(87) 




' .82 
(94) 


.72 ' 

(87) 


.71 

(68) 


.81 

(92) 


.67 

W) 




.84 


.7'4 
(87) 




.84 


.58 

(8?) 


.76 
(67) 


.73 


.81 

(8?') 


.67 
(67) 


.78 


.87 

(87) 


-.94 
(70) 


.97 

(94) 



Nu|ibi'rH in parentlinsQs nrn snmple ^l^es on which coGf f icients are ^ased, 
^-^Chdqge in ccmtent in fall 1976 (see text) p 
Change in scoring procedure for fall 1976 (see text) . ■ . /" 
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Measures that show Increasing reliability . Changes in 
content and scoring ^~ respectively , on Verbal Memory-1 and 
Arm Coordination have resulted in higher reliability Goefficients 
for both language samples, taking into account the apparent 
trend toward increased reliability of the measures in the 
spring. (That is, internal consistenp5^^^sten5ed to be higher 
in spring than in fall, so in comparimg spring 1976 and fall 1976 
coefficients, some ailowance must be^made for this trend,) 

Measures that show decreasing reliability . Only one scale, 
Draw-.5K]hild , fairs~ihtd this c^lass * As noted in the August 
1976 /Impact Study, Report, most children obtain near^maximum 
.scores on this measure as they approach ages ^ and 6; and as 
variance in item and total, scores diminishes^ so must the 
reliability cpefficient. An alternative to the standard 
McCarthy scoring procedure is available that is more sensitive 
to the finer differences that appear among the drawings pr^pduced 
by older children ^ and this procedure will be eKplored as a 
means of restoring reliability of the measure for children at 
levels higher than Head Start, . ■ 

Validity ' ' . . 

All of the instruments whose reliability was examined 
in the^ preceding sections show acceptable evidenoe of validity 
foruse with Head Start children, as discussed below* 

The validation procedures (described more fully in the 
Methods section of this report) involved determining the expected 
relationship of each measure with each other -one, then comparing 
these expectations * with the relationships that actually appeared 
in the data. Under this convergent-discrlminant method of 
assessing validity, the assumption is made that if an Instrument 
is actually measuring the construct it was intended to measure, 
the results will correlate highly with other measures of the same 
general construct^ will correlate moderately with measures of 
similar constructs^ and will not correlate at all with measures 
of independent constructs* Table 7 displays the matriK of ' 
exp^ctfd relationships* Tables 8 and 9 contain the actual ^ ^ 

correlations fo^= English- and Spanish-^dominant samples of Head 
Stat^t ^hi Idren , combined across groups and sites ^ for the fall 
1976 testing period^ (Note that the actual correlations are 
pre.sented for (som^ measures that do not appear in the hypothesized 
matrix-^^OCL subscales , height, and weight-^-because no expected 
relationship was stated.) 
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InttrgeErelatiQns of Child -Mgiiurti for English'Doniinant 'Simple^ 
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ilations of Child Meaiurts for Spaniih-Dorainant Sampli 
PDC Pall Data, 1976 
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Tii€ tAbrm donBistent the eKpect'ed and the actual relationships 
^are fox any measure, the stronger the basis for^assuming measure- 
ment^ validity * Under this procedure^ degree of validity is 
expressed in terms of the mean absolute^ deviatloTi of expected 
froni, actual relatp^onships p For example^ if it was expeated i ' 
that the correlation of ^Measure ^ with Measure B would fall in 
the range defined as "medium" B^t.it actually fell in the low" 
range, this /J%presented a de\/iation of'il^^unit from the 'expeatationk 
Deviations \^ere summed and averaged to' produce^ a validity index . *j 
for each meksure, ^ The deviations for the current testing pe^riod^ 
are sljown in Tables 10 and 11.' . ^ ^ ^ ^ 

/'Figure 7 presents profiles of the indexes so obtained for 
each measure at each of the three testing periods (fall 1975^ . 
spring 1976; and fall 1976) for the English- and ^Spanish-dominant 
samples of Jiead. St^t children. A mean deviation of 1 or leap 
was accepted as evidence of validity^ (deviations could theoretically 
range from 0 to 6) Accord'^g 'to this criterion^ ^11 the instru- 
ments examined are acceptably valid for Head Start children^ as 
evidenced by the stability of their validit^^ indexes across two, 
cohorts and three time^points, 

ChaTacteristics Determined/ from Past Reporting ■Periods i 
gensitivit^ tcT Change, Sui tab"! lity~f or Older Children, and . 
ReT-ationshrp to Social Competence ^' 

The preceding sections of thas report have dealt with the 
reliability and validity of the cnild measures based upon results 
of fall 1976 testing and upon earlier PDC testing, * Although the 
fall 19 7 6 data cannot be used at^ this time to examine two other 
critical test characteristics (sensitivity to change aAd suitability 
for use with older children)', it-Is appropriate at this point to 
review what we have learned. about these characteristics from the 
1975-76 testing periods. 

Since the impact Study will depend upr the PDC -battery 
of 'measures to* detect ^ change that ©an be a juted to program 
differences, several analysesH^ased on fail ±975 and spring 1976 
data were devoted to determining the extent to which the measures 
included in the battery are sensitive to change. The results 
of those analyses were considered' to be only estimates of the 
sensitivity that the measures are likely to show in the future* 
(In fact," the results are probably under-astimates because of a 
short f all-to^spring interval and some test revisiohs that took 
place be3tween testing periods.) 
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Deviations. of Qhild Meaiure Correlatiohs from Hypothesized Correlations, 
.jf- ' • ' Spaniah-Dominant Sample 
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Validity Profiles for Child Measures^ for two Head Sta^t 
Cohorts at a Jotal of Three Time Points 
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Spriffi 1976^ 



Fall 1976 



English Spanish Englisl^*f' Spanish 
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Note. The points-^^plotted represent the mean absolute deviation 
ofVthe measure from expected relationships with other 
mea^res. The broken line represents the level (1,0) 
above which deviations are considered excessive, (The 
range of theoretically possible deviations/ extend^e from 
0 to 6 . ) 

continued ' 
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Eigure 1 ■ 

Validity Profiles for Child Measures # for: two Head Start 

Cohorts at a Total of Three Time Points 
' ' ^ - (continued) * . ° 
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The sensitivity-to-"Change analyses were of three types* 
PJ.-rBt^ the ceYrelation of each measure^ in the fall and in 
the spring^ with child age tft . the tirr^e of testing was calculated 
to determine the age=relatedness of the measures (a substantial 
relationship was expected for most measures). Next ^ a paired 
(or correlated) t_ test was Galculated for the difference between , 
th^ fall mean score and the spring mean score on each measUre 
to ascertain if the scorea increased significantly from rfall to 
spring. Finally, a regression procedure was used to determine 
whether the observed spring mean oh a measure was equal to or ' 
greater than the expected, or predicted^ spring mean. This 
procedure would ascertain whether children gained at lekst as 
much as phBy were expected .to gain^ver the given time interval, 
'ihe re'sul-ts of the analyses were more critical fq.r some measures 
th^n for others. For example; scores on the Child Rating Scale 
anS the POCL were/expected to be less related to age than were ^ 
scores oh the other measurei, since the ratings were being made f 
relative to other children in the same age group, A^d while , 
these ratings were expected to change from fall to spring # they 
were not expected to change in a consisted up*or-down direction. 

Correlations with age . The . correlations of ea^ch of the 
^.measures with child "age" at the time of testing tended to be low, 
positive/ and significant^ with coefficientB generally around 
,15 to* ,30,< They had not, in fact, been expected to ■ be much 
higher, since the measures would not be useful for a program 
evaluation if they were related only to age rather than to 
differential experience , The only non-significant correlations 
(other than for WPPSI Block Design, see below) were within the 
Spariish^dominant sample, and although these correlations wer% y 
not substantially lower thar^ they were in the Ehglish=feominar^ 
sample, they were not statistically significant due to^^e 
smaller size of the Spanish-dominant sample, 

WPPSI Block Design showed a small negative correlation with 
age p This correlation was not expected to be negative/, but 
neither was^it expected to be significantly positive, [because 
it is presumed to be a measure of general ability^ a t^ait that 
is likely to remain invariant over short intervals. No ©thef^ 
sensitivity-tO'-change tests could be performed for Block Design 
because it was given at only one time-point. 

Fall-to-spring tahange : _ t tests . The Bilingual Syntax 
Measure^English showed a'~non--signif icant increase for Spanish^, 
dominant children. The sample sizes ^ however^ were extremely 
sm'all (5 and 4); so the measure may or may not be sensitive 
to change ^ (at least for this short a time period) when the 
language of the test is not the child's primary language. All 
other child measures showed a sigriificant f al l^to-=spring increase. 
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Fall^to-spring change^ regresBion ^analysis * ; For four 
of the child measures (BSM-E^ Draw-A\Child^ Verbal Fluency, " 
and Verbaj Memory^S) , values (repr^enting predictive powSr) 
^a£^ aomparable for fall and spring regression equations which 
J^relafred child status variables (age, sex, ethnicity preschool , 
^^^jperience , ^and sibiings) to child score on the measure • For 

of these Sour mea'sures, therefore, an expected spring score 
was calculated based on fall score , status on backgiroijnd variables, 
and spring__age . For Draw-A-Child, Verbal Fluency , and ^Verbal 
.Memory the actual spring mean was greaterr than the expected 
spring mean; i.e., the. children gained more 'on ttfe measures than" 
was expected as a function of theit iiicr^ase in age. For all 
four of the measures, more than half of the children obtained an 
aatual spring score that was equal to or greater than their 
eKpected spring score. These results imp1.y that the = tests are 
sen-si tiVe to change that is due to educatdpnal experience in * 
a^ition to experience that is , simply ^ function of ^ increased age. 
The fact that this analysis was not performed for the other 
measures does not mean that those tests are not sensitive to 
such change; they were excluded because the assumptions underlying 
the calculation of the predicted /spring score did not appear 
uenable for those particular tesos. 



^ Sumfnary of sensitivity to cnange . Basfed "on the results 
of the three analys¥s~discusied above , it was concluded that 
all of the child measures being used in the 19 76^77 PDC evaluation 
are in fact sensitive to change, within the developmental r^nge % 
represented in the i975--76 Head Start sample. f 



Suitability of the instruments for use in^the higher grades . 
Part of the task" of assessing the usef ulness of the child measures 
for the^ PDC evaluation has been to determine how suitable .they for use 
with children in kindergarten througji grade 3 as well as for use 

Head Start children, ^- During the 1975-76 testing periods, 
approximately 25 children per grade (kindergarten ttiirough grade 3) 
were tpsted in the Georgia site as part of the cross-sectional 
design there. In addition, 30 third graders were tested in 
Maryland. This information was used to determine how suitable the 
instruments can be ex'^ected to be for future use at those grade 
levels, as will be necessary if the evaluation is extended. 

Conclusions about the suitability of the child measures for 
use in- kindergarten through grade 3 were based on four factors, 
which were considered at each grade level: response" distributions 
on the items of each measure ^ ^ mean scores on each measure, 
reliability (internal consistency) of the measures, and validity 
of the measures. The criteria of acceptability for these four ^ 
factors were as follows^ 
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m Reaponse distributions ; An item was considered^ 
not to be useful at a given grade level if more 
♦ ^ than 75% of the childran at that level received" 
the maximum score for the item, 4 

m Mean scores ; Mean scores should, increase systematicflly 
across age levels^ eKcept for the rating scales, %ri 
addition^ if tl^e mean score a^, any level was grb^er 
than 80%' of the total possible score for* the Jtest ^ ^ 
the measure was faonsidered, unsuitable for chilsren of 
that age, 1 J . ' - ' . 

m Reliability and validity ; J: The criteria for these two 
factors were~essentially^he same as those applied to* 
findings for the Head St^t sample discussed previously, 

Base^/on these four factfers^ most of the measures appear 
to be useful through g^de 3^ either in their present foCTas or 
with modification. Each child measure is dioussed below, 

• Arm Coordinationf ^" All six items were useful^ based 
on response distributions^ from Head Start through 
grade 2, and only two items were not useful at grade 3, 
The mean scores increased acceptably across age levels 

. and did not reach a ceiling on the measure 1 The only 
indication of difficulty with this measure is in the 
validity profile > where the deviation from expected 
correlation levels is rather large at grade 2, However, 
the deviations appear to peak at grade 2 for all of the 
measures^ so this sample may not accurately represent 
the general population* Arm Coordination is therefore 
judged to be suitable for all grade levels, 

• BSM-English and BSM-Spanish i These measures will 
probablybe suitable for tke evaluation through 
grade 3, Intermediate graoes were not tested on 
these measures, so not enough information is available 
to make a definite decision regarding them* The mean 
scores and validity profiles are acceptable,^ The 
number of non-useful items (12 or 18 at grade 3 for 
the English version) and ^the third-grade internal 
consistency (alpha = *S8) appear to bes^problematic , 

• Draw-A-Child 1 Scores on this measure approach the 
maximum by grade 1, making it unusable in its present 
form beyond kindergarten* The problem with this test, 
however, appears to be in the scoring rather than in 
the nature of the task* We have reduced the complexity 
of the scoring from that recommended by the test author* 



ERIC 



53 



X 



We would now recommend returning to the more complex 
scoring or to a Goodenough--Haf ris type of scoring 
in order to make use of the valuable infonnation 
^ that the drawings^of older children can provide. 

With a <^hange in scoring procedures ^ Draw-A--Child ' ^ 
is eKpected to be suitable for use across the age . 
/range that will be spanned by a /longitudinal study 
of PDC. ^ / , ; ~ 

# pips I Based on all four considerations, the PI^S 

is judged to be suitably fpr "use at all grade levels 
^ of the evaluation, " ^ 

# POCL and Child Rating Scale i Both rating scales 

are judgedT to' be suitable for use at all grade levels* 
Mean scores, alphas, and validity appear to be / 
acceptable . The response distributions are acceptable, 
but tend to be unusually centralLy distributed for the 
POCL for grades 1 through 3 (which may be attributable 
to the particular testers who assigned the ratings — this 
will be investigated in the future)* 

m Verbal Fluency i Basea on all four considerations. 
Verbal "Fluency is judged to be suitable for use at 
all grade levels, 

m Verbal Memory- 1 ^ This test produces scores very 
near ceiling at Head Start, and last spring was ^ 
found to be unusable beyond kindergarten because 
most of the children received the maximum score* 
Accordingly, for fall 1976 testing two items were 
added to the original four; the added itemsi were 
^constructed on the models of the original word 
strings^ but being longer, present a more difficult 
task* Logically^ this should result in a test 
* celling that will ,be reached by fewer older children, 
but this proposition has not been tested* Among 
this year's Head Start children, though, the internal 
consistency ^of this scale was higher than it was 
for last year's Head Start children. 

# • Verbal Memory-3 i Based on all four considerations/ 
Verbal~Memory- 3 is judged to be suitable for use 

at all grade levels to be spanned by the evaluation. 

Rela tionship to social competence * Since the PDC battery 
was constituted with the intent of providing for measurement of 
the traits that comprise social competence , •an analysis was 
performed for the last Impact Study report that examined the 
relationship of spring 1976 test scores (for Cohort 1 and higher-grade 
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Bamples) to ad hoc ^riteria of social competence * The 'criteria' 
ware established by factor analyzing ratings from the PDC ' 
Child Rating .Scale and the Pupil Observation Checklist {POCL}^^ — j 
instrmtients completed^ by each child's teacher and tester > . 
respectively^ — ^then^ on the' bdsis of the results^ creating^<;^ 
.factor scores ,for eaoJ#Ghild that rep»san±e^his ^ he^ ^^atus 
on each of the "social'*comi3etence" f actors. The Child rating 
Scale and POCL Were chdsen *a^s #6urces of the proxy criteri^ 
because the assessments, prpvided by the teacher and tester^ 
arW based upon observationsif of each child's behavior, in 
variety of formal aTid informal situations , and thus logica|i|^ ^ 
come close to representing measures of the child's -everyday 
effectiveness^" i-.e.^ social competence, ^ | ^ 

- . * • ..f ' ■ . . — -^y/ 

The object of the analysis (a linear regression procedure) 
was to determine ^he pagnitude of the relationship existing, 
between the testy included in the^ PDC battery *and the "soaia^^ ^ 
competence" criteria. The more relevant the tests are to ^ 
social competence, the stronger the relationship expected. The 
tests that entered into th% analysis included BSM-English, Acti 
Coordination, Draw-A-Child, Verbal Fluency, Verbal Memory-^1, 
verbal Memory-S, and the PIPSV. All of these eKcept Arm 4, 
Coordination were found to be substantially associated .with the 
collective "social competence criteria i values ranged from 

.1\7 for BSM-English to .37 for the PIPS; for Arm Coordination 
wffs .05* For children in the higher-grade sample (consisting 
of pooled K-3 samples) a significant relationship was found 
for all^ the tests except DrawrA-Child (whose low reliability' 
for older children has alreadf been noted) and Verbal Memory-1 
(which ,>^e fore this year's revision, also showed low .reliability 
among ql&Ler children). Arm Coordination was significantly 
related tp the "social c6mpetence"^criteria for this sample 
(r2 = ,12), perhaps because of ;the"greater variance occurring 
across, the brpader age range. Verbal Memory--3 and the PIPS . 
showed the strongest relationship to the predictors for this 
sample; both had coefficients of .25. 

* -■ 

This technique, aithough quite exploratory, produced 
results that constitute at least preliminary confirmation" of 
expectations: all the aforementioned tests , originally 
selected for their theoretical relevance to social competence, 
seem to provide" measures' that- are empirically relevant to 
social competence, or to our best approximation of that 
construct. " 



Ease of Admi nis t rat i on 



It IS of little use ta select tests which may have all 
' of tfie necesskry psychometric qualities if they cannot be 
administered sa.tisf actorily by the testers hired in each 
PDC c^fflfcunity. Therefore/ one of the factors, taken into 
c^iside^tion when tests were being reviewed fo^ the PDC ^ 
Imj^ct Study was their general suitability for ^^administration 
by paraprof essional . Even though all of the tests selected 
f ^DC mat this requirement/ there were some differences in the 
ease/dif^ icuTty of their adjninistration . 'The details presented ■ 
ielow axe based on observations of tester performance during 
the*, tester trainfng session and on the continual feedback -by 
testers throughout the ^ata collectipn period. (See Appendix B 
for the ^ moni tor ing forms used'^as checks on compliance with 
testing specifications.) ^ 

Bilingu ar Syntex. Measure (BSM) . This test is relatively, 
easy to administer-^^the directions are straightforward and 
the cartoonlike pictures usually capture the children's attention 
and elicit responses to the items. The. only problematic area ' ' 
of the BSM administration is recording the child^'s response^ / 
accurately. Testers have to listen rcarefully to th6 child's" 
answer and 'record it verbatim. .This has to be a conscious , 
effort since \^duits tend to write a ^-rrect verb tense or word ^ 
ending automatically whan the, child Ms used incorrect tense 
or wording. " . : 

Verbal Memory . This test presents few administration 
problems. In Part 1 the tester slowly reads "a string of words 
and records those the chi Id repeat S/ while, in Part 3 (Part 2 
is not included in the PDC battery) the tester reads a story 
and records the child's account of it. The only administration 
e^rror noted/ a minor one f is a tGndency on the part of some ^ 
"testers to paraphrase the sta^ndard'- encouragements "rather than 
repeat them exactly. / ^ " - 

WPPSI Block Design . -More time v/as/spent practicing this 
test than any of. the oThers beciuse of the lO^^dif f erent designs 
the tester mus-t learn to cons truct witk the blocks, and because 
of the leng^bby instructions. The tester has to learn to make 
the designs wiile simultaneously' reading^ aloud the instructions 
> for const ructing ' the de^^gn. However, once familiar with' ' • 
construcbion of the designs .tosterB can usually coordinate 
bntfi ,u^t ivitiMS. Th(^roaftor, nnly minor ^worcllng errors tend 
to furcMir duriru-j Lha actual tost admi ni s t ra t J=on , ^ 



Dmy^-A-Chi Id . ' No f^roblems wore Qncountered in the admini' 
H tf^ntiiorT^f thi J tnst. Most of f Jio * crh i. Idren nnjoyeti it and it 
wa^;i good loaci-in t:u3t ror the? socuiid ■ :.^)n . 



Verbal Fluency , This test is easy to^ administer. The 
taster, instructs the child to name all of the toys (or things^ 
tp eat or names of people, etc) he can within a specific 
time period. Only a minor problem was discovered^-'a tendency 
among testers to paraphrase the encouragements. 

Preschool Interpersonal Problem-Solving Test (PIPS) * 
Most of the administration problems encountered with the 
PIPS were related to SGoring,/ The object of the test, which 
presents the' child with hypothetical interpersonal problems, 
is to have the child suggest as many different solutions as he 
can think of (e.g., =If A has the truck and B wants to play with 
it, what can B do to get a chance to play with it? If C has 
the sailboat and D wants to play with it, what can D do to get 
a chance to, play with it?). Once the child responds, the tester 
has to decide whether or not the^response is relevant and 
whether or riot the child has given a similar answer already. 
If the^ response is new and relevant^ the tester must decide in 
wh^^ of 16- scoring categories to place the answer (e,g,, ask , 
share, trade , bribe ) . Many of the ^ responses that children, give 
are clearly understandable such as, "ask him for it," "share it," 
"tell his mother," but other answers require more judgment on the 
part of the tester and, as a result, soyie of these are scored 
incorrectly. Therefore, tester s need to have a clear understandin 
^of each category and how it differs from the others. 

Arm Coordina'tion , Although appearing simple and 
straightforward ,¥his test has the greatest potential for 
presenting difficulties in administration *since the tester has 
to attend to so many details at the same time* For example, in 
Part 2 (beanbag catch game) the tester must coordinate reading 
of the instructions with tossing the beanbag while also watching 
to see if the child stepped* over the line and if he used the 
correct hand to catch the beanbag* The tester then must record 
the child's response while makii^ sure the= child isn't tossing 
the beanbag to her** Since the tester has to attend to 'many 
details during the test it is easy to commit wording errors or 
to overlook some of the necessalry activities* 

Pupil Observation . Checklist (P OCL) . Some testers had 
problems in differentiating among children when completing this 
rating scale. That is, some tended to- rate every child "average" 
on alii of the dimensions* For example, even though the instruc- 
tions indicated that a child should be rated "average" on the 
"Cooperative-Resistive" dimension if the child was as cooperative 
or as .resistive as other children during testing, it seems 
unli^kely that all chilflren would behave identically during the 
test sessions. More specific rating scale instructions may be 
needed fior those testers who tended _ to rate all children the same* 
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Summary . Generally^ .^hen^ the tests are not difficult 
to administer. Tester perforrrtance improves with practice 
and the administrat^ion difficulties mentioned in this section 
are more/apparent with new testers than with eKperienced ones. 

L 

■ ■ L ■ 

Factor Structure of the Battery 

Factor analyses^ were performed to investigate inter- 
relationships among the child measures in the fall 1976 battery. 
Separate analyses were performed for the English^dominant and 
Spanisrf-dominant samples, with the results shown in. Tables 12 
and 13. While parallel versions o'^ the same measures were 
included in the two analyses, the s -ibstantial difference in ^> 
sample sizes ^English - 880, Spanish ^= 85) constrains ' the 
expected comparability of the results. In view of that limitation, 
the resulting 'factor structures of the total battery for the 
separate samples are remarkably similar. 

Results for the English-dominant sample . Analysis yielded 
three distinct factors v>rh~ich,^after~rotation, accounted for* 
26,7%, 19,3%> and 11,6% of the variance , respectively , The 
scales that loaded highest on the first factor, which gould be 
labeled "verbal-responsive," were Verbal Fluency ^ Verbal MemoryTl# 
Verbal Memory^3 , PIPS, and POCL,^ The BSM-English also loaded 
substantially on this factor, but loaded higher yet on the 
second factor. The scales that loaded highest on factor 2, 
which could be called "cognitive f leKibility / " were BSI^-English , 
WPPSI Block Design, and Draw-A-Chi Id * The^;:r|hird factor appears 
to repraseilt a "psychomotor" dimension* Oj^Sy Arm Coordination ^ 
had a substantial loading on this f actor , ;^:Al 1 three factors 
combined accounted for 57.6% of the variahcp.:ampng measures 
for the English-^dominant sample, " ■i.^'^. 

Results for the Spanish-dominant sample ir Factor analysis 
yielded "three distinct" factors" which , aftet' ^dtation , accounted 
for 13%, 26.1% and 22.3% of . the variance , 'r4:spectively . On 
the first factor , apparently representing the dimension "psychomotor," 
only Arm Coordination had a substantial loading. On thp second 
factor, which might be designated " verbal^cognitive , " the scales 
that had the highest loadings were WPPSI Bloc^ Design/ Verbal 
Fluency, Verbal Memory-1, Verbal Memory^3| and Draw-A-Child . 
On the third factor which could be* termed " verbal^-resonsive , " 
the tests loading highest .were BSM--Spanish ^ PIPS and POGL* 
Combined, these three factors accounted for 61.4% of the variance 
amang measureB for the Spanish-dominant sample. 
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A factor analysis of the POCL itself is presented in App 
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Factor Analysis of Scores on Child Measures, 
English-Dominant Head Start Children, 
Fall 1976 Data 

N^880 



■3 


Factor Loading of Child Measures^ * 
(highest loading ' italicized) 


CHILD MEASUBiE 


. Factor 1 


Factor 2 


Factor 3 


BSM^English 


*41. 


.48 
.82 


^ 

.20 


WPPSI Block Design 


.^9 


= .17 


Verbal Fluency 


, ' .59 


.36 


-.15 


Verbal Memory- 1 


. 70 


"04 


• .01 


Verbal Memory-S 


.71 


•19' 


. . 01 


At^rtt Coordinati"^ 


.08 


.05 


-.96 


Draw-A-Child \ 


.17 . 


.80 


.03 


PIPS \ 


r .6T' 


.14 


, 03 


POOL 


• ¥ 


.11 




^Principal components 


solution, varimax rotation 
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Tabl© 13 

Factor Analysis^ of Scores on Child Measures, 
Spanish-Dominant Head Start Children, 
Fall 1976 Data 

N=8 5' 





Factor Loading of Child Measures 




(highest loading italicized) 




Factor 1 


Factor 2 


Factor 3 


BSM-Spanish 


-.23 


.35 


'.60 


WPPSI Block Design - 


.13 


.78 


.04 


Verbal Fluency 


-.09 


, 63 


.35 










Verbal Memory-1 


.01 


. 63 


.42 


Verbal Mernory^S 


-.4 5 




'.4 0 


Arm Coordination 


.as 


.17 


.03 


Draw--A-Child 


. 05 


■ . 75 


-.01 


PIPS 


-.14 


.11 


.?£■" 


POCL 


. 28 


.01 


. 79 



Principal components/ solution , varimax rotation. ;. 
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Comparisons with previous factor analyses' ^ The results 
of the Current anaTyses are not directly comparable to the 
spring 1976 analyses/ since Child Rating Scale data (for '^the ^ 
English=domirxant sample) were included at that time, Howeyer^ 
there are interesting similarities. Arm Coordination maintains 
its position as a distinct dimension^ aifd language tasks and 
the social problem^solving measures (PIPS,' POCL) are clustered 
in a similar manner. The resetablance between the factor .structures 
foulid for the two language samples recoimnends the possibility of 
equating English and Spanish versions of the ba^^^ties in the 
future . 



Characteristics of the Classroom. Observation System 

The PDC Classroom Observation System differs in many 
ways from the other instruments in the battery of child measures i 
it is no.t a test of perforntance under special conditions but 
rather a record of performance under natural conditions; it is 
conceived as reflecting characteristics of the classroom environ-^ 
ment as much as it reflects characteristics of the children who 
comprise the class; its scoring and the methods hy which its 
reliability and validity are established^ are unique to this — 
measure among those in the battery* Thus in this^section the 
Observation System is discussed separately from the other 
instruments* . : 

The PDC Observation System ^as developed to provide 
doscriptive in^formation regarding the social-^emotional competence 
of children in their classroom settings. The behavior categories 
that make up the instrument were formed by redefining, and in 
some cases, combining, behavior categories from existing observa- 
tion instruments that dif f erehtiate between children^ of varying 
degrees of social compe'tence, and by adding other categorie,s 
appropriate to PDC goals. The theoretical rationale for 
selecting these categories is that they measure a "general attitude 
of negotiation ' and reciprocity in dealing with others in a social 
environment."^ This attitude is believed to be generalizable 
across all cultural groups and impldes 'that a child's own needs 
and goals are valuable, but that the needs and goals of others 
are equally important and must be taken into account; More 
speciMcally, the developing child should leari^^ow to control 
and influence others with effective' strategies tff&Jb ^o not 
violate the rights of others. (For example, physical force is 



- Bronson , M, Executive competence in preschool children . Paper 
presented at t^he meeting of the American Educational Research 
Association, Washington, D.C, 1975. For a more e-xtenslve 
listing of references to the literature consulted in developing 
the system, see interim. Roport II , Part B (June 1975), 



considered to be a violation of others^ and thus^ioas not 
indicate an attitude of negotiation and reciprocity*) In ) 
addition, the child shduld be reasonably influenced by others ^ 
but not totally subservient to or dominated by them* Other 
social strategies that promote and-sustain social interaction 
such as sharing, helping/ requesting and providing resources ^ 
and taking turns , are also considered important indications of 
a child's social competence and are represented in the categories 
of the observation system. Definitions and eKamples of the 
observation categories are presented in Appendix F. 



S uminary of Instrument Development 

Fall 1975 and spring 1976 observation data collection 
efforts were aimed at establishing the psychometric properties 
of the observation instrument* After each collection, pbservers 
reported that the instrument could be used in the field with 
li-te-tle difficulty. In addition, spring reliability assessments 
have shown ^^hat observers can be trained to use the instrument 
with a desirable level of accuracy. This implies that observation 
categories have been sufficiently defined and clarified that 
minimal inference is required by the observer when coding 
specific behaviors. 

Analyses oX^f^H 1975 and spring 1976 observation data 
have shown some ofegree of relationship between children's social^ 
emotional% psychomotor, and cognitive competences, i . e children ' s 
obsorved social skills corresponded to their performance on 
other measures in the PDC battery . But in view of the low magnitude 
of these relationships, par ticularffly with regard to tefichers ' 
ratings of similar dimensions of children's behaviors, the 
validity of the instrument as a measure of children's social skills, 
could not be adequately established. ' - 

i 

In part, this absence of validity mayv be . attributed to the 
in which behaviors were sampled. gincffi the activity level of 
the classroom (i.e., opportunity for social) interactions) was \^ 
found to be highly related to children's behaviors, it seems 
likely that the 20-rninute observation period provided a 
description of the child's behavior under only one particular 
classroom condition. Under other conditions, the child -s 
behavior might have been very different from that sampled . 
The time-sampling technique, then, could account for the low 
correspondence between a teacher's assessment of a child's 
social skills and the description provided by the observation 
rnstrumont, 

* .... 

For this re an on i. r: wciB propOBOc; in fnuDrim Roporfc IV ^ 
{Aucjust 1976) /that tho Observation Systura can ba rcigardeci as a 
measure of classroom "personalities," rather than of individual 
behaviors, and that it. be used for aBscsslng PDC impact at a 
c'Lassroom level of analysis, 

Q " ' b 2 ^ o 



^ Purpose of this Analysis 

^his analysis is directed toward establishing t^e 
psyphom&tric properties of the Observation System as' a P 
measure of" classroom characteristics* One step in this 
process is to substantiate the findings that observers 
can be trained to use the instrument with acceptabla 
levels of accuracy. Thus^ as in the past analyses, j 
reliability data "were collected and analyzed for this 
report. A second step is to assess whether the instrument 
accurately represents ^ . and thus adequately measures , tthe 
classroom* This was determined by pairing observationl 
subcategories with other measures in the PDC battery and 
^assessing their relationships using classroom=level analyses. 

It is also important to establish the value of the 
instrument for measuring PDC's;impact on children's classroom 
behaviors or on classroom conditions. Exploratory analyses 
were thus conducted which examined the comparability of PDC 
and comparison classrooms in fall 1976, No consistent ■ 
differences were expected^ since this is the first operational 
yeair of PDC/ but these analyses were intended to provide a 
check on the initial equivalence of PDC and comparison 
classrooms on such dimensions as frequency of social inter^ 
aci^QUf amount of time spent in nonsoci;^^ activities/ amount 
of verbal behavior/ and opportunity for%^ocial interactions. 
These comparisons also provided a test of the analytic^ 
methods available for investigating classroom comparability, 

Ob s elevation Procedures 



Jn an attempt to insure that observation- data would be 
. collected in a consistent manner' across siteS/ guidelines 
and procedures for completing observations were specified 
in detail during the observer training session, 

^ / ' 

Before they began their observations in the classrooms / 
* observers met with classroom teachers, to describe the obser-^ 
yation instrument and answer questions. To control ^for any 
observation biaS/ the observers completed all- observations 
prior to administering the child tests and observed only 
f- h n r h i 1 d r n n who we ro list cd o n the i r r o s t e r s , 

1 U/' t j i ri 1 1 i r 1 q w i. t h tlx u firs L child o n their r o s t e r s / 
ribsGrvors observeBd, each child for two consscutiva five--minute 
intervalB. Each five "minute interval was divided into 
fifteen 20-second units. These units were further divided: 
5 seconds for observing and 15 seconds for recording. The 
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observing and recording intervals v/ere sign£illeci by a 
portable cassette tape recorder that emitted an electronic 
"beep" into an earphone worn by the observer. 

• The number of qbservations complctGd per day varied ^ 
with the number of cMldrert in the classroom^ the class 
schedule for that par'ticular day, children ' a absences , 
and activities that took 'the children outside the classroom. 
Observers were advised to observe during cAl periods of 
the day except outdoor play and toileting.^ If, however, 
regular classroom activities such as storytime/ art, or 
^snacktime were conducted outdoors, they were instructed 
to observe during those times as well. Obser-vers also 
received instructions on how to handle situations that 
might interrupt their observations ■ and on how to handle^ 
child absences. 

Fall 1976 Observatio n Training Pro cedures 

The primary objective Of the SeptembeT 1976 training 
session was to adequately train observers, especially new 
ones, so that reasonable coding reliability could be achieved. 
To the extent possible, only those observers who had 
received training and , collected observation data in fall 
1975 and/or spring 1976 wera included in this training 
session, ^ This allowed new training methods to build upon 
the group *s previous training and experience In the classroom. 

As in other observation training sessions, the Observa= 
tion System was introduced in a larger-group session during 
v/hich changes and revisions were noted and examples of 
observation categories were provided. Small--group sessions 
v/ere scheduled to explain and give examples of the observation 
categories* Throughout these sessions, observers were 
asked to describe and role-play examples of behaviors they 
had observed in p^revious classroomi observations; trainers 
than indicated how) the behaviors should be coded on the 
recor^d sheet. Additional small-^group sessions were used' 
for viewing videotapes of preschool-^aged children in school 
settings. After the observers had coded a two--minute 
segment of the tape, trainers provided feedback on how the 
behaviors should have been coded. Common errors made by 
the participants were discussed and additional clarification 
and examples were pro\^ided for ambiguous or frequently 



: L'/o ■ )i jse r^^ : r.^ wh'> '.'.''.r- "o rjollect 

fall i 9 7 F> • oijso r'/a tion data .:iad rvoith'.^:^ attended other 
obsor'/ation training i^ossion?^ nor I'^ntod nbservation 

djta rjr':!v Lo iB 1.'/, , 



4 

Reliability of the Observ ation System 

Collection of reliability data .^ The reliability estimates 
included in this report were' 'gathered at the end of the fall 
1976 training session. This ' reliability assessment was 
necessarf for determining how well observers were prepared 
to begin\their observations in the field. . To assess this, 
observersj' simultaneously watched and independently coded a 
40-minut/ videotape which included several clear examples of 
the behavioral categories contained in the observation system, 

Aif^ysi s of rel iability data . The accuracy of observation 
codin^^as assessed by comparing each observer's responses to 

cri^^rion coding of the same behavioral events. Although the 
measure produced is not identical to a conventional measure of 
inter-observer agreement, it does assess the accuracy of 
observers' coding compared to a single standard criterion. This 
provides a basis for detecting those categories that were commonly 
coded unreliably by a majority of the observers* Further^ 
analytical inferences that include these categories could take 
this " "^^^ into, account, 

ethods for ^sessing Qoding reliability were eiAployed. 
The f method computed a pairwise observer and criteriJDn 

^^agre- enc estimate within categories for each ■5-second ol:|servation 
inte al* h proportion of agreement was determined usinc 
Cart-^ ight's alpha, ^ This procedure consists of comparimgj 
unit by unit, the codes selected by the observer with the 
critorion codes* Estimates were obtained for the number^^f 
Lime;, observer and criterion codes agreed and disagreed for^ 
eac observation unit. The reliability was then computed by 
dividing the number of times codes agreed by the number of 
agreements plus disagreements, ^ 7 

The second method of reliability assessment examined how 
well .-^servers' cq^es matched the criterion for the total length 
of a given observation period. (Although the observers' codes 
may not agree with the criterion unit by unit, it is important 
that observers, after viewing a child for a specified interval, 
at least agree^ on the relative number of tallies assigned to each 
subcategory.)' To obtain this estimate, the mean total number of 
tallies assigned to a given category was divided by the criterion 
numljor for that category (i,c,, the number of times behavior of 
that type was actually exhibited in the reliability tape) . Dif^ 
ferent inferences can be drawn from these two reliability estimates 
t>he first, based upon Cartwright's alpha, indicates the reliability 
of a B_ing_le^ observation within a specified category; the second, 
overall proportion of agreement, indicates the reiiiabili-dy of the 
total frequency of observations for a category. 

/ 



' C a r t w r 1 g h t , D , £ . A r a p i d , n o n = pa r amo t r 1 c o s t i ma t ea o f mu 1 1 i = 
judqe roliability. Psychometr ika , 19 b6, 2_]^, 17-2 9. 
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Reliability results . Table 14 presents the mean proportion 
of agreement and CartwrTght's alpha for each subcategory of the 
observation system. Also included in this table is an indication , 
of whether observers overestirnated or underestimated the frequency 
of a specif ic subcategory. 

As has been found in past analyses, it appears that 
observers can accurately distlnguish_and cod'e children ' s involve- 
ment in the classroom ( Noninvolved f Social , (and Nonsocial ) . 
Proportions of agreement for these' subcategories ranged from 
-96 to ,99, with" a mean of .98. These reliability estimates 
are higher than those reported for these subcategories in spring . 
1976. The high agreement figures, ,99, for verbal behavior 
( Veroal English and Nonverbal ) ^ found in the current analysis 
are "^Iso Ino'tewbrthy . ' ^ 

V while there i^ a slight increase in reliability estimates 
froln spring to fall, errors coVitinue to occur most frequently iS; 
in xhose subcategories describing the child ^s behavior during 
chiid^peer and child-adult interactions. Proportions of agreement \ 
in these subcategories ranged frojn .57 to .98, with a mean of j 
.81. However, considering the five distinctions that these / 
categories require observers to make concerning the nature and / 
purpose of the child's social interactions, some imprecision is / 
expected. Thus the coding accuracy figures seem acceptable." 

Items that describe classroom activity ( Maximal , Moderate , 
and Minimal ) were excluded from this reliability assessment because 
of drfficulties in port ray ing the general activity level of a/ " 
classroom \^)n videotape. However, iast spring's onsite reliability 
assessment ' indicated that observers could accurately assess and 
code a child *s opportunities for social interaction in the classroom. 
For that assessment, thekmean reliability estimate £or this 
category was .88. Because ^this category was not revised, it is 
assumed that this coding, accuracy was maintained for fall data 
collect ionv. 

Overall, there was a substantial improvement in observers- 
observation and coding skills. In comparing fall and spring 
reliability estimates, the accuracy of coding increased or 
remained the same for 80% of the observation categories. 



AnaJy:.Mii of Classroom Observa t i_on__D^i ta - 

P repara tio n of observation data for desc riptive an alysis, 
For aTl obstirvation categories, a sum of the child's behavioral 
incidents across the 10-^minu.te observation intervals was computed, 
Each child, th*::n^ had ,ono summary scorn for overy item on the 
nhv;< va t- i on i n ^a. runinn . ThoBO scoros v/^m^^ t^hrni HUinmod across 
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children within classrooms > creating a classrobm mean for 

each category. Because past observatiori ^analyses indicated 

that children's behaviors varied accoi^ding to the activity 

level of the classroom, a classroom scoi-e should take this 

into account. Therefore, as in past analyses, relative ' 

frequencies were computed for each activicy levels weighting 

the absolute scores by the amount of time classrooms were 

observed in a given activity level. The means and standard , 

deviations of these transformed variables are given in AppendiK F. 

Results of descriptive analyses . In order to provide a 
summary description of the "obse^rvation data, the classroom 
relative frequency for each observation variable was computed 
for Bach activity level. Classroom frequencies were then aggregated 
across sites. Figure 8 presents the relative amount of time 
children, in these classrooms were engaged in Noninvolved , Sociaq , 
and Nonsocial activities. The results " indicate""€"hat th¥se groups 
spent high proportions of their time in activities that involved 
objects (46%), and in activities that involved persons (48%). 
These proportions were found to vary across activity levels with 
Minimal activity levels proving most conducive to social inter-^ 
actions. Interactions during Minimal activity periods, however, 
often occurred in a large-group setting where Ghildren were 
attending to the adult leading the group activity. These social 
interactions differ qualitatively *^rom those that would occur 
during free^play or small-group time (i.e., Maximal and Moderate 
classroom activity levels). ^ ^ 

Figure 9 shows the relative frequencies of children's 
verbal . behavior for those classrooms in which the majority of 
children spoke English. Again, as in past analyses children 
were verbal less than 30% of the time, with some variation across 
activity levels. Out of the total time children were verbal, 

of this verbal activity occurred during peer interactions, 
v/hile 48% occurred during adult interactions. 

Figure 10 shows the relative frequehcies of children -s 
vGrbal behavior for Spanish-speaking classrooms. For these 
classrooms, ch '^-en were verbal less than 40% of the time, 
with only sligh /ariations across activity levels; Spanish 
was spoken 21% of the time and English . was spoken 17% of the 
time. Out of the total time Spanish was used in social interactions, 
6 6'^. of. this verbal behavior occurred during peer interactions, 
whilo 34". DGCurrcd during adult interactions. There was no 
<\ \ i i ''v^^nci^ i'n ( hf^ ' 1 n t i vo amount' of time?' nhildren spoke English 
{20':.) or H}.janisli (20-k) during adult in toract idna ; however, 
UurMu.j . r 1 n Le racM:, 1 ons Span LtBh way spokon moro Erequtmtly 
tjhan En'jUsh (13'Vv3. 27^0,. " " ' . 
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FigurdpS ' ^ . ^ 

■ ' ' i X- \ • 
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by Classroom Activityv Level ^ 
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modei^ate^ and mirM^mUm opportunity fori social in'teractions . 
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Figures 11 and 12 display the relative frequei^cies of 
child-peer and child-adult classroom .interactiona . ^ Classroom 
social interactions more often involved adults (50%) , Peer 
interactions occurred 45% 'of the time, while joint interactions 
with 'both a peer and adult occurred 5% of the time* As in past 
analyses, it appears that the majority of children' s social 
interactions with peers and adults were positive in nature> 
as negative behaviors were eKhibited less than 1% of. the time. 
The rates of children's positive attempts to Control peers 
(33%) or adults (34% ) : are Ijigher than any previous analysis 

-'has shown. Adult Controls fluctuated substantially aGroas 
activity le^^ls; peer Controls ^ were only slightly influenced 
by the activity level of the llassroom. Further ^ the relatively 
large standar# deviation (,20) of these proportions suggests ^ 

, that there is some variability across plassrooms in the frequency / 
of controlling incidents. [^., ^ / . / 

Classroom behaviors reflecting children's positive attempts * 
to resist the control of others (i,e*> Assert ) occurred infrequently 
2^ to 4% of the time. Because the occurrence of this behavior is 
associated with the amount of control exhibited by .others ^ the 
low frequenc]^ of Asserts behaviors^ implies either that children 
we're not. directed by othets (which is contrary to the evidence*^ 
of increased controlling irt the classroom) ^ or that they have not 
learned acceptable ways of resisting this control* Thua^ it 
appears they are resisting control in a ne'gative^ manner (coded 
Negative ) or simply complying with others' directives (coded 
Positive ) . ^ However , last spring's observation data refl^cte^fe^ 
higher f requency of assertive behaviors, which suggests /^hat the 
new sample ^of children may learn these strategies , as they gain 
more exp^ience in Head Start classrooms. 

Figure 13 describes child-adult and, child-peer interactions 
fjfom another perspective i purpose of interaction rather thari^ 
. nature of inter act ion'. High propagations of Gives "behaviors were 
exhibited . J During adult interactions/ children provided infer- ^ 
mation 4 5 to 59S^of the time, and provided assistance4or materials 
6 to 8% of the'±jLme. A high . incidence of Gives Inform ation 
(48% to 51%) ana^ Gives Assistance (.7 to 10%) was also found for 
peer interactions. In contrast. Requests behaviors were exhibited 
i n tho cl assroom 1 eBS; than 30% o f ^ he t i me * These behaviors 
wur o primarl iy di roc tod toward obtaining assistance or materials 
from aither an adult (131) or peer (17%); requesting or providing 
emotional support ocow^^red le^ss than 11 of the time^ for both " " > 
adult and peer intera-exions . Classr6om patterns of Gives and . , .. 
Requests varii|||^only moderately across activity level's* ' There 
was^ howev^, some evidence that as activity levels changed ^rom 
Maxim'al to^i^inima-l conditions for social interactions , Gives 
^fori na t i^on .. (wi th i n the context of adult interactions) increased . 
BecauHu ihstrutetional activities (e,g.^ music^ storytime) typify 
Moderate and^ Minimal conditions ^ it is not surprising that^ the 
in#idsnce of providing information increased for these' activity^ 
levels . y-- • \ " , ' " 
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' child-PQur interaction^^ represent 45% of classroori^ social activities 

^^Parcen tage indicate^ the relative, amount of time classrooms were , 
observed under^ condrtions that respectively ^permit maximum/ ^ 
moderate, and. minimum opportunity for social interactions. 
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.. . Results of correlational anaryses . The correlatf ons 
shown in Table 15 were computed to assess the interrelationships, 
among olDservi^tion variables at the classrooTn level. ^ For this 
analysis ^ . particular attention was given to determining wheth^er 
findings 'from past analyses hold for classroom^levei data or 
whether new xeiationships emerge, . ' ^ \ 

As in past analyses, children's rates of verbal behavior 
and the strat^ies they use while interacting with others were 
only slightly related. This finding suggests that, fQr this 
young age group, children control others and resist the control 
of others in a nonyerbal fashion . Also, children often request 
assistance from others nonverbally, as a significant correlation 
was found between the Nonverbal and Requests Assistance categories 
Other signifi'tant correlations were f ound" between children ' s 
Gontrolling^ behaviors and the frequency of Requests Assistance . 
Thus, as has been found in past analyses , children 's c~ontrolling 
behaviors were generally directed toward obtaining help or 
materials from an adult peer.; that is, they wpre directed . V 
toward some end o^ goal J, ^ 

A departure from past fadings, though, is the significant ' 
intercorrelationi between peer and adult interaction items 
were foHnd in this analysis* In classrooms where, there was a 
high frequency of peer Controls , there w^ 'also a high frequency 
of adult Controls . A similar relationship was found between peer 
and adult requests for assistance* . 

Of additii^al interest are the^ relationships between 
observation variables and other child measures. To examine 
these, classroom means were computed for the child tests and 
correlated with observation variables shown in Table 16, 
the relationships found ranged from moderately negative to 
moderately positive. Observation categories that occurred more 
frequently (e.g.. Social , Nonsocial , Verbal English , Nonverbal ) 
tended to show higher Gorrelatl6n"s~with other va"r~iables".~" 



^For'tfiis analysis, children's scores on the observation variables 
were averaged within classrooms. The classroom then became the 
unit of analysis. 

- Only classrbojns in.^hich the majority of children spoke English 
were used in this analysis. 
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j^oif^iii' points have bet^ftonlttod Xraxorrelatio^^ co^iffiaieiitSi 
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UF^:^n/'.ni: J:i v:ii'^:: ^.h^ liiajorltv 'if dhiiiren Enfdiih w-ro iis^d in thlE analysis. 



Although behivior in the individual subcategories 
describing peer and adult interactions in the clasBroom 
occurred less frequently, some of these categories w^ra 
moderately # and in some cases significantly^ related to 
other child measures* In particular, rates of peer and 
adult positive behaviors were positively related to scores 
on the PIPS. ^Additionally , rates of requesting help Jrom adults 
or peers were significantly but negatively correlated with BSM/ 
Draw-A-^Child, and WPfrSi scores. It appears that the rate of 
requests for assistance is yMrersely related to a wide range of 
children's cognitive and lai^^age competencies. 

The findings from this analysis indicate that a number- of 
olassroom--level interaction patterns are related to performance 
on psychomotor, cognitive^ and language measures* Inasmuch as 

f^higher degrees of association were found for those categories 
describing global characteristics of the 'classroom (e,g,, 

. Involvement , Verbal Behavior ^ Rate of ^ Adult Interactions ) , greater 
confidence can be placed in the findings related" "to" these categories. 

Results of comparability ana lyses. The assumption made in 
identifying certain Head Start centers and schools as comparison 
.institutions that they are essentially the same to begin with 
as their PDC counterparts. Therefore ^ since last fall marked 
the start of PDC's fi^rst operational year--its official 
beginning-^-'the classrioom observations made in the fall were 
expected to indicate that no systematic differences exist between 
PDC and comparison classrooms* To confirm (or disconfirm) this 
expectation, analyses w^^re performed that examined the measures 
obtained for PDC and comparison classrooms in ten categories 
with the highest incidence of occurrence. These were ^ate of: 

Noninvolvement ^ 
Social Involvement, 
Nonsocial Involvement, 
V 

Nonverbal Behavior, 
Pear Interactions, 
Adult Interactions, 

Maximal, Moderate and Minimal Classroom Activity- 

Two alternate types of analyses,^ parametric and non-parametric, 
were conducted; the results of one provided cross-checks on the 
results of the other* Both procedures were consistent in showing 
no significafnt differences between groups, with one epcception: 



^The analytic procedures that led to the results described here 
.are discussed in Appendix H. 



the non^parametric procedure 'indicated that children in' 
Gompa^ison dlasses spend a significantly greater^ proportion 
of their time in nonverbal activities (the parametric procedure 
produced a finding of no significant difference on this variabl'^ 
Considering the fact that a total of 20 significance tests were, 
performed (10 variables x 2 statistical procedures) a finding 
of only one significant' difference indicates that PDC and 
comparison classes are indeed very much the same on the . ^ 
dimensions measured by the Observation System. 
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IV 



: CONCLUSIONS 



Adequacy of the Tests and the Samples 



ThQ major purposes .of this report are^ as noted, to 
answer the questibns of. instrument appropriatenees , PDC^ 
comparison group comparability, and sampie sxzm adequacy. 
The answers to be offered here are tentative in two waysi 
they anticipate the conclusions ^hat will emerge from the 
meeting of the PDC Evaluation Advisory Panel in4. April, .and 
they anticipate the conclusions that will be drawn by OCD,. 
which bears the ultimate responsibility for decisions made 
about PDC, ' ^ / - 

Are the Measuring Instruments Appropriate tp the Task? 

Child measures / individually . It can be said^ with " 
. few reservatrons that" all of~the instruments included in v 
the battery satisfy all the criteria that have been used , . 
in judging them. As Table 17 reflects, the reservatioiis 
are minors Verbal Memory-1 and Draw^A-'Ghild were earlier 
found inappropriate for older children but, with changes 
in the co^itent of the fdrmer and the scoring of the latter r 
it is believed_that they will now be appropriate; the English 
-version- of- the^iili^iguai;^^Synt^ 

sensitive to change for childrer) whose dominant language is 
Spanish, and the. Spanish version may not be adequately 
sensitive for English-dominant children. 

■ . ^ ■ '- ■ ' 

' ■ i id measures, collectively . Although the factor 

^structure" of the battery does" not correspond exactly to the 
a priari ca togori nation of the tests (cognitive'-language ; 
socTaT-irnotional , psychomotor), the factors that ehierge are 
similar to those expected for both the English^dominant and 
Spanish--dominant groups, and indicate that the battery does ; 

^ provida coverage of these areas, ^ 
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' V The^ PDQ Ola^room Obssrvatlon System, ' The Obse.rvmtlon 
System was ^not exainiSed^a^^ sama crttferla, ussd 

; V for thm other tests,, pjLrtly because it is-cQnsidered to \ 

providTe measures^ of group/ rather than indivicjuar, behavior. , 
By thp criteria developed /for evaluating the dbservatibn - 
y/ System, nece^s than those usedfor the 

more Qonvention^l instruments, it appeal ' acqept^le as 
. a means of /assessing the diassropm environment^ This / ^ ^ 
^po'teatia,! application makes the instrument ^particularly ' , . 
^ useful^ since it is the classroom .that' mediates between 
PDC-^lnduqed aaminlstrativf change ^ (measured in the ^u ^ * 

of the Implementation Study) *and the program's^ intended ' v 
^ r ef fects on '^chil The / ; ' ^ " 

' - ' Obseryatibn System providfs ttie, pnly w^y of examining , 
eveiit's at^ thls^ critical mediating ^tagp^ with the 

cancellation of the -Tea'cher ,Sui^ey , this Junction, beboftes 
.even more critical. ^ ' * ' ^ , , , ;y 

^ Other measures w -The Bpring" 197? battery will include 
aWthe^a foremen tipned measures plus height and weight . ^ 
(included as -preWeasures ih^this fall ' s battery) and 'th#. , 
. PDG Child Rating Scale, which was not administered . in the 
fa^ because the^ teachers^ho complete it c§nriot be eKpected 
fully acquainted ^with the children that early in the / . 

a] ^ 

/ ■• . ■ ■ ^ ' ■ ^ ' . ■ ^ / 

, Are the PDC and Comparison Groups Really Comparable j ^ i 

.. . - - - ^ . \ ' " . ^ . ^ " \ 

Overalls findings . At the site level. Cohort 2 PDC . 
. anfi' comparison grdups seem to ^be more similar than were 
Cohort 1 grbups on the criticM dimensions of ethnicity, 
prior preschobl experience , and spcioeconomic status^ (repre- ' 
/ ^sented by mother ' s education and number of siblings) . For ^ 
'half thelites, there 'were no signifiQant PpC=comparison 
differences at all on either the background characteristics 
or the tests. For the other half of the siies, one or two, 
differences each were found, but what; s mora 'important is 
that the PDC^ and cOTiparigpn groups in these sites were \ 
.found ^o be alike on*the remaining 18 or so dimensions^ 
'examined. The aggregated English-domipiant PDC and comparison 
groups are quite comparable on all the crit^al background 
characteristics; the groups differ on only one of thirteen 
child measured* and on none of the six bfiekground variables* 
The aggregated Spanish-Dominant groups do not differ on any 
o€ the^child measures, but do^ differ in^prior preschopl 
eKperience. ; . 



'■23 ' . ^ 
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I. Prospects/ foy- ana^^^^^^ a.t the aggregate l&vel . The 

coinp6siti6n of the aggregated English^dominant PDC and * j . 
.coinparlipn g^ups seems quite satisfactory for analytic ' \; 
purposes. in future analyses of test score ga,in, the one 
variaBler 61^ which^ th# groups differed, initially can be 
adjus^ied Without difficulty to make allowances for differencaa 
in initi^ status. 

aggre^ted Spanish-dominant groups also show con^ 
siderafele similarity^.*: differing significantly on only one 
of -19/ dimensions examinied. But that one is prior preschool, 
expe^^ience ^ factor that might* well be Qxpected to make a 
dif/ference in children's adjustment to school life, thus 
poisibly obscuring whatever effects PDC may produce. (Only 
11% o{ the Spanish-domiftant PDC children attended preschool 
^before' the present Head Start year, versus '39% ^of the 
Spanish'"dominant compkrison children.) Still, despite this 
speculation, it may turn out that the difference in preschool 
experience will actually ^be of little cgnsequence (both 
groups , ^ after all, will have had at' least one year of Head 
Start), And even if^rior preschool experience J.s found 
in future years to affect the performarfqe , of these children, 
certain statistical -strategies can be appidfed when, analyzing 
PDC *s^ effects to reduce the bias introduced by the differing 
preschool experience of the PDC and comparison groups, 

^' Thus considering" the two major possibilities^^ (a) that^^ 
the difference , in prior preschool experience may ^ave no 
future biasing^ effect and' (b) that .even if a biasing lef feet 
is discovere^d it can probably be minimized 'Statistically==it; 
is recommended that the evaluation proceed 'for the Spanish- 
domijiant sample. This aspect of the evaluation ^ it should 
be noted ^ is of 'special interest, since bilingual/bicultural 
education is orie of PDC - s special emphases* 

f Prospects' f or^'ana^lysis at the site level . The value 
of performing ^analyses of child impact ^at the site level 
is questionable f^or two reasons* First, there are indications 
of scime initial group differences at abo^ut h^lf of the sites. 
Second, there may be less to learn from twelve separate 
site-level ^analyses than there %s\ from analyses that examine- 
groups aggregated across sites; this is so because site-i'#Vel 
analyses cornmand less statistical power and because it is ' 
difficuTt to draw generalizations from f indin^^_.that may 
flue tup to unsys toma tical ly from site to si£e* ' 



Will Large Enough Samplas of ^^^C^^ Remain- in PDC' and 

Comparison Schodle^ Site to Permit a LongltuQlnal 

Study ofProgram Effects? • 



■ It is obvious from attrX'tion^rojections that if the ' / 
evaluation depehded upon site-leve^ analyses :o£ PDC* s effecis 
on childrenr the sample si^es available at most sites: would , 
be inadequate by the time- Cohort 2 /reaches. first gr 
i^ is possible that as soon as nexA year the number of 
Cohort 2' children remaining at PDC Wd comparispn schools 
in some sites will ie too small to pVrmit stajfe^tically 
adequate within=site analyses. By aggregating PDC and 
cbmparison groups across sites, howe ve f^ --a ^ su f f i c lent 
sample can be cons»tituted 'to allow analYSes to continue 
through 1980-81, when Cohort 2 will be in grade 3. This 
is certainly true for the English-^dominant sample, 'at least* 
it is;ipBs certainly true for the Spanish-dominant; sample. 
However^ even for the latter^ analyses could proceed for a 
few years — long enough to allow preliminary conclusions 
to be drawn about the effects of. PDC. ^ ^ - , 



The, issue is largely one of costi are the projected 
final samples of English-^doittinant PPC^and comparison 
children — numbering about 200 and 170, respectively—large 
enough to justify the expense of the Impact Study? The 
questiqn cannot be answered fully within this- report---a 
wider forum is required. 



Prospects for Testing OtD' s Hypotheses 
Regarding PDC's Impact on Children 



CCD's original speci 
included three hypotheses 
impauct on children; since 
written a fourth hypothes 
these hypotheses, which a 
the major mission of the 
tudinally. 



fications for the PDC evaluation 
c^cerning .the program' ^ expected 
the time when that docurfient was 

is has been developed* 'Resting of . 

re ^recapitulated below, will be 

Impact Study if, it continues longi-- 



Head S hart-'Entry PDC^Group vs. Head Start-Entvy 

: Children ^who) enter a PDC program 
t level will show significant gains 

social 'competence battery) over 
ren who enter a non--PDC program at 
evel* . - . 



Compavtson Group 
at the Head Star 
(on tests in the 
comparable child 
the Head Start 1 
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"■ Hmad Stavt^Entvy FpC Group vs.. 'Kind&^gar^&nSntry 

P.D*C Group Chiidren who enter a PDC JprogrMi - at 
^the'Hf ad Start level will show signiiicant gains 
^^var comparable ags'-mal^es who enter- ^he PDC 
, ^g'rogram in kindergarten^ not having attended Head 
; '-S^rt. '. ^ ' \.. . : 

3 , _ -Ki^ergarten-Entry PDC Group vs. Head Start^Entrg' 

Comparison iroupj _ Children who enter. PDC. at the ^ . 
' kirtMe^garten level will jhow significant gains over 
comparable age-piates whd^^enter a. non-^PDC Head Start. 
\^nd pjogrees through a non=-PD^ elementary school. 

4. - KtndBt^^rten--Ey%try PDC Graup vs, Kindsrgart&n^Entry , 

Compar%^Qn Croup / Children who enter a PDC 

program i% the kindergarten level will show , "'^ i 

significant gains^over comparable children who 

enter a, comparison program at the kindergarten ; level j 



\ ' 



Hypothesis 1 is PDC's cornerstone, and up to the present^ 
the efforts of thp Impact Study have been directed solely, 
toward determiining the feasibility of testing Hypothesis 1, 
The conclusions drawn in the preceding /section affirm that 
the hypothesis can Indeed by tested with a reasonable 
expectation of obtaining decisive. results • Thus it is 
-appropriate now to consider the feasibility of ^testing 
Hypotheses 2, 3, and 4 * ' , 

Testing of Hypotheses 2 and 3 wil\l require identif icatioh 
of a sample of children who (a) resembae Cohort 2 PDC \ 
children on educationally relevant diminsions, (b) have^ 
had no Head Start experience, and (c) inter PDC at the 
kindergarten level* Testing' of Hypothesis 4 will required 
identification of another group of children' who satisfy 
conditions (a) and (b) and who enter a Comparison school 
at the kindergarten level, 

-The feasibility of testing these hypotheses depends 
upMDa ' the chances ^of .^being able to locate\ children who are 
like Cohort 2 children on the dimensions \ eKamined ^ in this 
report (e.g, , ethnicity, ' family backgro\^nd) , who would 
have- been eligible, or nearly eligible, to attend Head 
St.irt, but did not, and who atte'^nd the same Schools as, 
Cohort 2 children./ These conditions present ^^several problems, 
edumerated below, 1 . ■ ^ 



(1) It is far from certaih that such children exist in 
numbers sufficient for statistical investigation , 
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/(2) I'f there are enough such children, it ^ill be 
difficult to^ identify, them Jbecause the information required 
to establish socioeconomic status ^ w^ich. is one of the ' 
major matching criteria, is \often inaccessible ^ i^ 
accessible only at great coslt in time and erf|ort-| 

(3) If the children. do exist and are ideri?tl|FiS^^ 
problem remains of establishing a baseline for their 
performance , oh the Jneasures J.n the PDC battery^. 



The third point calls for amplification Hypotheses 
2 and 3 require that children who enter a PDC p^^gram in 
kindergarten be compared with children of the*s^me age who 
entered' a PDC (or comparison) prbgram at the Head Start 
level- . The mechanics of the analytic procedure . in this * 
case will actually involve comparing the progress of one 
group/ of children with the progress of the other. 
And to measure progress , it is necessary to*determine eafch 
child'*' s basiline at the onset of the program. For Cohort 2 
children / the progr^ began at age .4 with the Head Start 
year,, and each child's performance baseline was determined 
by , administratron of jthe fall 19.76 bat^ry. But since the 
kindergarten^entry chiltoen cannot ^^^identified until they 
enter school at age 5v it will not be^^posaible to establish 
an age' 4 baseline for them directly* Of course^ it is 
possible to use^ ^ge 5 as a baseline for both groups^ bflt; 
the risk in this is that PDC or Head Start will already ^ 
have raised the performaince level of ^heVb^o^p.^2 children, 
and treating age 5 as th$ starting point will/render the 



^-^rogram effect invisible. 



. This discii&sion is not intended 'to discourage a/ 
decision to^^pursue the testing of Hypotheses 2 through 
4; its purpose is to delineate tlil.is"sues that must be 
considered before such a' decision is made* Final- 
recommendations ^on this and other matters will be 
presented to" OCD* immediately after the April meeting of 
the PDC Evaluation Advisory Panel, 
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* Descrlptlona of the Meaeuras in ^ the t^all Battery , 



Order of _ " ' 
A&ninistration ^ 



Social-Emotional Measures 



PDC Classroom Observation Sj^s tern ^ ^ ^1 

Preschool Interpersonal Problem-^ ' 

Solving Te^t (PIPS) ; 7 

Pupii Observation. Checklist-. (POCL) 9 

Psychomotor Measures » " ^ 

^ I 

Arm Cgordinatipn [McCarthy Scales of 

Children's ^ility (MSCA) ] \ ^ 8 
Cogni'tive and Language Measures 

- - ^ ' ^ . 

Bilingual Syntax Measure' (BSM) ' 2 

Block Design (WPPSl) . 4 

Draw-A-Child (MSCA) 5 ■ 

Verbal Memory (MSCA) . 3 

Verbal Fluency j (MSCA) - . 6 



"^ Other Measures 
Adult Language Check 

Attrition^ Handicap and Attendance Inftir- 
mation Sheet , ^ (~ 



Each of these measures is^ described briefly below. For 
a more extensive review^ see Interim Report II ^ Part B: 
RecQnmendatiQns for Measuring ^Program Impact (1975), 



-As noted in the text, the battery was administered in two 
pr^ some times thrae sessions. 
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PQC Classroom Observation System (High/Scope. Foundation^ 
. unpublished) . Thg PDC observation system was developed t5. 
provide information about children's classroom behavior 
K along dim^slons pertinent t^ the social^emotional goals 
of Project Developmental Continuity The system focuses— 
on aspects of an individual child's behavior> verbal or 
•nonverbal, that reflect the child's attitude toward himself , . 
and on the child's social competence as demonstrated in- - - 
his interaction wit'h pe^s and adults. ^ ; ' 

Using a tim^ .sapling method, trained observers obserjve 
each child for five minutes at four different times during., 
the day and code their behfevior into 'four general categories^ 
"noninvolved, " "involved, " "interacts with peer," and ^ 
"interacts with adult, " A fifth category, "activity l^el,". 
is included to provide information concerning the context 
' in which these behaviors were observed* Each- of these 
categories includes subcategories ^hat are designed to 



^identify the frequpncy and 
within the general ca^e^or 



nature, of specific behaviors 



general category, ^ 

■ - t ■ ' -. - = 

Preschool Interpersonal Problem-Solying Test (Shure and 
Spi vack , 1974p , The PIPS attempts to assess the child's 
ability to name alternative solutions to a life-related 
problem-'-that of obtaining a toy from another child. Paper 
cut-'OUts of boys, girls and toys are used in presenting 
the problem. Among, inner . city four-year-olds attending ^ . 
the Philadelphia Get Set day care program, those judged 
as better-adjusted by thei> teachers were able to concept- 
tualize a greater number and a wi^er range of alternative 
solutions to real-life, problems than were their more poorly 
adjusted classmates » 

/ P upil Observation Checklist (High/Scope Foundation / 
unpubrished) . This is a raking scale consisting of twelve 
7-p"oint~bipolar ad jectives /derived from a similar scale 
used in the Home Start evaiuation.' The tester rates each 
child using this instrumertt after he or she has ^ administered 
all the other measures in ^the battery to the child. See 
Appendix H for data'ils on ^ the factor structure of this 
instrument. ^ . ^ 



^Shure, M. B, & Spivack, G. The PIPS Test Manual , Philadelphia: 
Hahnerrian Medical Gollege, 1974, ^ , . ' \ 

2 Love J-,et al. National Homg Start Evalua^tion ^ Interim 
Repcrrt VII. . Yps iTantlT MlV Hirih/Scope Foundation, March 1976. 
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flcCarthy Scales of Children's Abilities (McCarthy i 
1972)^ * These subtests consist of . a sefieg of tasks tapping 
problem-solving r psychomotQr , and 'conceptual abilities^ ' 
and are similar to the Wecrteler scales, but with emphasis 
on age-related maturationar indicators, 

• ^ * r ■ ^ - • = 

- • Verbal Memory . The. child is asked to repeat 

' sequences of wor ds^ J Verbal Memory-' T) and to , 
repeat or retell as much as possible of a one 
paragraph story" (Verbal Memory^3) , .. - =^ 

Verbal ^ Fluency . The child is asked ^ to '^ame a's 
many" meipfiers pf specific categories (e,g%^ animals) 
ms he/she canr ^ ^ 

Arm Coordination , Child bounces a r^fcber' ball, . 
catches a beanbag^ and^hrows a beanbaf through 
a hole in a target. 



Draw-A-^Chi-Ld , Child draws a picture 
thQ same sex, ' . 



Wechsler Preschool- and primary Scale 
Block Design subtest (Wechsler^ 1967)^, 




a child of 



Intelligence / 



task rjequires 

reproducing ( cons tr uct Ing ) designs with ffat colored blocks 
either from the examiner's model or from a picture on/ a ^ 
card.. The measure taps prdbrem^solving abilitieSi flexibility 
of response . style ^ visual=motor organization ^ and executTOnp 

^ Bilingual Syntax Measure (Burt, Dulay and Hernandez-Ch, ^ 



1975: 



This test' is designed to measure children !b oral 



proficiency in English and/or Spanish standard granmatical 
structures, Simple questions are used with cartoon-type 
colored pictures to provide a conversational setting for ^ 
ellGiting natural speech. An analysis of the child 's^ res- 
ponse yields a numerical indicator and a qualitative des- 
cription of the child's structural language proficiency in 
standard English br ptandard Spanish^ Responses are written 
down verbatim. The English version is administered only to 
English-dominant or bilingual children; the Spanish ; version 
only to^panish-dominant or bilingual childten, 

^McCarthy, D, McCarthy jScales of Children -s Abilities- ^ Maagal 
'New York^ Psychological Corporation^ 1972 * - 

'^Wechsler, D,^ Wechsler Preschool and Primary Scale of Intel" 
ligenc^: Manual . New YorkT "Psychological Corporation^ - 1967 . 



Measure . 



Dulay, H. 
New York; 



& Hernandez^Ch * 
Harcourt r Brae 



E . Bilingual Syntax 
, Jov^novich^ 1975. 
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Adult Language Ghecki . thisvmeasure is used in the 
bilingual/bicultural demonstration sites to obtain an 
indication of the. languages the adults in'^the dtaeeroom 
use. during thieir interactions with children* The inter* * ' 
viewer sits In the classroom'^for a two=hour=^eriod and 
records the language usedYby the teachers and aides v. 
approKimately every five minutes . The Adult .Language Check 
was used only in classrooms where languages other than 
English ^ere . spoken by_tea^^ or other adults* 

Demographic Information Sheet . Additional ^^nforn. 
mation about each child in the sample /'feuch as prevlousV : 
preschool experience^ handicap status^ *dominaht language^ 
etc* is obtained from, Head Starl: records. 

Height and Weight . -All .children are Weighed and 
measured during the same two^week period in the fall. 



Controlling for Order Effect in Administering the BSH ^ 
to BlTlngual Children - • ^ . ^ ■ 

Children who show facility in both Spanish and 
English receive both versions of the BSM, THe order in 
which the two versions are administered is^co^rolled 
,so that during . any single testing period half the phil^ 
dren receive the^ Spanish~"version first apd half receive 
the English version first* Further/ the order is 
reversed with each successive .testing so that/ for 
example, a child who received the Spanish version first 
in the fall would receive the English version first in 
the spring* . ; 
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Forms for Weekly ' T©:ater Monitoring 



0 



r 



The fgrms reproduced here were, used weekly by testers 
"for mutual monitoring . The completed forms were 'returnee 
regularly to fiigh/Scope for. continuing analysis. In this 
^appendix, the cat.egories beside which an X appears are 
those in, iA?hich testers, as a group made more' errors than 
expected or than was judged tolerable* 



Tabie B-1. , 
ARM COORDINATION 

Monitoring Form^ 



Inte^rviewpr / 



Date 



s form will provide High/Scope Foundation with information on hovvf 



Child's* Nama ^ 

INSTOUCTIONS: \itii 

simiiar the intiewie^w administrations are within each'' site and across sites. The 
*intarviews must be administ'ered in. a ^standa^d uniforrn way to insure, gompa 
of the data. Wh^ you monitor another interviewer you should be^ recording the child's 
responses in your interview booklet and be watching for and noting wffetKer any of the 
following errors occur during each of the interviews. You will fill outgone of these 
monit^ing forms ,-for- each/ in ^ef view you monitor 



4l 



Test Administration E-rrors 



3, 

4 . 

5 . 



Fails to have CORRECT INTERVIEWl^^G MATERIALS; 
m.g.f didn*t haye ball^ beanbag , tape , etc* 

INCORRECT PLACEMENT of interview^ materials ; 
e.g.,, didnVt have target from child, : 
didh*t kneel or bend when throwing beanbag 
to child , etc . . 

INCORRECT WORDING of^ interview questions; 
e*g.^ doesn't follow the wordsv in the inter- 
view booklet. 

SKIPPED AN ITEM. . 



SKIPPED A SECOND TRIAL ^ or gave a second 
atrial when- it should not have^been given. 

6 . STOPPEb 'iNTERVIEWs INCORRECTLY ; e.g., gave 
Part B in Beanbag Catch Game*when child 
fa'iled to catch in Part A, 

7^ REPEATS; repeated the interview question 
more than one ^tifne. ' ' 



8: 

10 
11 



ENCOURAGEME^^TS^; gave more 'than one encour^ ^ 
agemeiit V®r d^ltial question ^and repeat; 
didnVt give an encouragement when needed* 

SCORING; scored child *s response incorrectly. 

INCORRECT TIMING; failed to mark i#me . 
stopped test booklet, 

OTHER: tspecify) j 



Check Each Time 
Etrdr Occurs 



Rapport with chil^ (circle one) 



Poor 



Adequate 



Good 



' ■ Table 1-2 . ■ 

BILiNGUAL SYNTAX MEASURE 
Monitoring Form 



Interviewer 
Child's Name 



Date 



INSTRUCTIONS; This form Will provide High/Scope Foundation with information on "how 
similar the interviev^ administrations within each site and across sites. *The . 

interviews . must' be itoinistered m a ^andard or uniform way to insure comparaisility 
of the data. Whm yoU monitor another interviewer you should be recording the child's 
responses in your interview booklet and be watchrng for an^inoting whaler any of the 
.follo>/ing errors occur durinf each of . the interviews. You^will fill out one of ^ase 
monitoring forms for each interview you monitor.^ . , 



Test 



listratipn Errors 



Check Each Time 
ErrQr Occurs 



Fails to have CORRECT INTERVIEWING MATERIALS; 
e.g., i-s missing the warm-up picture. 

JNCORRECT PLACEMENT of interview , materials ; 
e.g., dbesn^^t place warm-up picture directly 
in front of child, ^ doesn ' t place" picture 
booklet directly in front of child; didjn^'^t 
^put warm-up picture out of child ' s sight when 
using booklet, etc. 

INCORRECT WORDING of interview questions; 
'e.g., doesn't follow the words in the ' 
interviaw bpoklet; adds too many aHditional^ 
cpmments or^ .'questions . 

- ■ ' ■ ' 

SKIPPED AN ITEM. " 



5. STOPPED INTERVIEW INCORRECTLY; e.g. 



didn 



stop. after dte'm 5 when child responded to 
only two^>©f .the first five items didn ' t 
stop af£er^ four DK^R-NR. 

INCOBRECT TIMING; e .^g . d-idn ' t mark time 
star^ted and t^^nie stopped, on cover of 



interview-, b 



et . 



/ 



10 



P^PEATS ; repeated, the /inte'rvie^^ questoion 

more than one time or(^dn't r^epeat the . 

question when it should have been repeated; 

reDeated the child's response verbally, 
" * , r ^ ■. , 

ENCOURAGEMENTS ; gave more than one ©ncour- 

agement after the ini tial que^stion ; gave^ 

more than. one encouragement after, the repeat 

or didn't give an .encouragement when it 

should have befji given. ' ^ 

SCORING; not wrrting child's response, 
exactly as said; nfft* writing legibly. 

DEFINES WORDS; defining word^s for child ' 
during the non^preliminary questions. 



11, OTHER: 



ispeci f y 
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O Rapport with child (b,irc4.0^ one; 



Sobr 



9H < 




Table Bt3 
,,DRAW-A-CHILD' 

orm. 



Interviewer ,.■ 
Child'/ Hame 



'Monitoring F 



/ 



Date 



INSTmJCTIOTSs Thim form will provide High/Scope Foundation with information on how 
similar the intervfiew adndni strati ons are within each site an(^ across sites* The 
irviews must be administered in a ftandard or uniform way to insure comparability 
the data. Wheri you monitor another int(Srviewar you should be recording^ the child's 
^responses in ^y our interview booklet and be watching for aifd notitig v^etiisr any of the 
following^ errors occtir during each of the intefview§* You will fill out^ane of these 
saonitoring forms for each interview you monitor* ^ ' * ^ - ^ 



.Test Administration Errors 



CKeak Each Time 
Error Occurs 



1, INCORRECT PLACErffiNT of interview mater i,- 
a,g.^ didn't, pi ace blank page widthrw^i 



in front of child. 



2 * *INCOR^'CT WORDING of intervrew questions ; 
, . e * g * , doesn * t follow the words Lth^thm inter^ 
view booklet. /^^ 

3. INCORRECT TIMING; e^ g . , didn^V.^ mark time 
started. : 



4,. REPEATS; repeats the interview question, 



5. ENCOU^GEMENTS; failed t^ give one 
encQuragemfent &pe^ified ^a,n test 
booklet when needed. 



6. OTHER: . (spdcifyl 



dp" 



Rapport with chijd (circle one) 
Name of Mbnitor ; 



Poor 



Adequate 



^ Gdoc^ 



ERLC 



9 9 



Hi 



. Table B-4 ■ 

. PIPS •• ; . ^. 

Monitor.ing Form / 



Interviewer 



Child's Name 



INSTRUCTIONS! Tliis form will prbvide High/S^pe Foundation with information on how 
similar the interview administrations are within ^ach site.^nd across -sites. The 
interviews must be administered ijx a ^tanda^^d tiiifotm way to insure com^arabd li ty 
o^ the d^ta, When^you monitor ar^other /^interviewer you should be recording the child's 
responses in ya^ir interview booklet and be watching for and noting whether any of^ the 
folLowirfg- errors occur ^during . each pf the 'interviews. You will fill ou^ one of ttvcse 
mDnitoring forms for each interview you/ftionitoti. j 



■ % 



Test, ^dntinistratibn 



I 



1. Fails to havG CORRECT ■ INTERVIEWING 'MATERIALSI 
e.g., 'missihg^ one .of tKe PIPS cutouts, etc. 

2. INCORRECT PLACEMENT. of interview materials; 
e.g., putting toy o^li wrong cut--out/ placing 

\ cut-outs on table rather than on some kind 
of stand. . . . 

3. INGORRECT XORD/_NG tof interview questions; 
^e . g . /^o0Si^^^"^^foliQw the words in the 
^interview fcooklet. 

4* SKIPPED/AN ITEM. 



5. STOPPED INTERVIEW INCORRECTLY; e.g.,^didn*t 
stop interview after two donsecutive stories 
■ irt*^li5ich ^ahild ^gave .repetitidn ^qE . ari's^^arsr' no 
r solution answers, or DK^R-NR, ^ v ^ ^ ' 



6, PROBING, too many or too few; e*g., didn't 
probe when response required it or probed 
when child * s answer was acceptable. 

7. SCORINQV recorded .child; s response 4 
incorrectly; failed to put chll^d's-* 
responsG in correct response box. 



Check teach Tinle 
. Error Occurs . 



8. OTHE'R: 



spcci f y 



ERIC 



Rapport with child (circle' one) 
Name of Monitor r 



Poor 



Adequate- 



Good 



1017 



1 



Table B-5 
VERBAL FLUENCY 
Mbnitoring Form 



Intarvii 

Child! s Name 



Date 



tmsi ^is form will provide High/Scope Foundation with Information on how 
similar the interview administrations are within each site and across sites. .The 
i interviews must be admnistered in a standard or uniform way to insure comparability 
of the data* Whei you monitor ^another interyiawer you should be recording the child' i 
responses in your interview booklet and be watching for and noting Aether any of the 
following errors occi^ during each of tlie interviews . You will fill out one of thasa 
'TOnitQrin^ forms for each interview -you monitor, ' > - } * 



Test Administration Errors 



Check Each Time ^ 
Error Occurs 



1, 
3- 



INCOR^CT WORDING of interview questions; 
e*g. /ji^eBn ' t^ follow the words in the 
inte^iew booklet, 



5. 
6. 



SKIPPED AN ITEM. 

STOPPED INTERVIEW INCOR^CTLY; e.g., 
didn't give entire interview . 

INCORRECT TIMING; e*g-, allowed the child 
more than or less than 20 seconds to name all 
the toys^^lip.^ could think of (tester and moni- 
tor sHould^ be within 5 seconds of eaqh other 
on th6 timing) • . - 

REPEATS; jripea ted the interview question. 

SCORING; didn't record child ' s response ; 
exactly as- said, didn't write legibly ^ 

ENCOUM.GEMENTS; failed to say appropri^e 
encouragement after 5 seconds when it should 
have been said, or enqourag.ed too many times. 



8. 0TH6R* (specify) 



Rapport with child (aircie one) : ^ 
Name of Monitor . 



Poor 



Adequate 



Good 
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Table B^6 , . ' 
. VERBAL MEMORY 
Monitoring Form 



Interviewer 
Child's Name 



T 



Dat^ 



INSTRUCTIONS i 'THis/ form will provide High/Scope Foundation with ; information nr\ how 
similar the intarview administrations are, within each site and across sites, Tho 
intarviews must be ad^nistered in a standard or uniform way to insure comparability 
of the data. Mien you mpnitor another interviewer you should recording the child's 
Responses in your interview booklet and be watching for and^ noting whether^ any of t'ho 
following eri'ors oceur dur'ing e^ch of the interviews, a You will fill out one of these 
monitoring f^rms for each interview you monitor^ " . • 



Test Administration Errors 



Check Each Time 
Error Occurs 



5, 
6\ 



9, 



INCORRECT WORDING^ of interview questions; 
e*g.> doesn't foirow the words in the 
: interview booklet. 

SKIPPED AN ITEM, 

STOPPED INTERVIEW INCORRECTLY; e*g,, failed 
to stop Part I after child gave no correct 
answers to items 2 and 3, or items 4 and 5; 
gave Part. II when shouldn't have, 

INCORRECT TIMING; didn't mark time started 
on test booklet. 

REPEATS; repeated the Interview question, ' 

SPEED; read the words too quickly for the - 
child or allowed too much time between tjie 
words. 

^ENCOURAGEMENTS; encouraged the child n^re 
than once or didn ' t -^encourage the child ^ at 
all when he^didn't respond didn't use 
specified encouragement, 

SCORING; failed to record chi.^d ' s response ^ 
correctly or wrote child's response 
illegibly, . , 

-OTHERS (specify)^ . 



X 



Rapport with child (circle one] 
Name? of Monitor 



PQor 



Adequate 



Good 



EKLC 
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' Tabic B-7 
•WPPSI BLOCK DESIGN 
Monitoring Form 



Interviewer ^ 
Child's Name 



Date 



INSTRUCTIONS: This fo^ni will provide High/Scope Foundation with inforination on how 
similar the interview "administrations are within each site and across sites. The 
interviews must be administered in a standard or uniform way to insure oomparability 
of the data. When you monitor another interviewer you should be recording the chird*£ 
responses in your interview booklet -and be watching for and noting whfther any of the 
follQwing errors occur during each of the interviews- You will fill but one of these 
monitoring forms fox^ each interview you monitor. 



Test Administration Errors 



Check Each Time 
Error Occurs 



Fails to have CORRECT INTERVIEWING MATERIALS i 

e,gl/ doesn't have all' 14 blocks, doesn't 

have picture booklet, 

"' "1" ^ - - ' - . - . - 

INCORRECT PLACEMENT of interview materials; 

e,g,, makes incorrect WPPSI design, uses 

wrong blocks in making design / giving wrong 

blocks' to child. ' i 

INCORRECT WORDING of interview questions; 
e,g.^ doesn't follow the words in the 
interview booklet. 

4. SKIPPED AN ITEM, ' ^ 

'5. SKIPPED A SECOND TRIAL, or gave a second' 
trial when it should not have been given* 

'6. STOPPED' INTERVIEW INCORRECTLY; ,e,g. , ^ didn't ^ 
stop after two consecutive failures, or didn't 
give item 4 after child failed items 2 and 3, - 

7. INCORRECT .TIMING; e.g., allowed the child more 
or less time to make the design than the in-- 
structiQns indicated (tester and- monitor 
should, be within 5 seconds of each other on 
the timing) , didn ' t mark time stopped, 

8. REPEATS; gave a demonstration when it should 
not have b,een given or failed to give a demon-" 

. stration; rapeated the interview question. = 

9. ENCOURAGEMENTS; gave more than one encourage- 
ment or none at all with the initial question 
and. more than one encouragement ^or nona at* all 
on the second trial. 

10. SCORING; scored child's respon-se incorrectly, 

11. ROTATIONS 'and GAPS- failed to correct child ' s 
rota'tions or ask child/ '*Is that right?" when 
he left more than 1/4 inch between his blocks* 



12. OTHER : 



;fepecif y] 



Rapport with child' (cimle one).: - Poor 

C. : . ^ : ' J I . 



Adequate 



Good 



( 



_ ■ J. ■ • APPENDIX C ' . 

Commentary ori Scor'ing the McCarthy Arm Coordination Scale 



a 
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. ■■' Commentary on Scoring the ? 
. McCarthy'**Arm Coordination Scale 

Followin^^i^e "fall 1975 date collection^ a nujftoer ^ 
of alternative procedures were explored £or scoring items 
and scale scores i^^e^chl Id 'test battery , Particular 
attention wasfcaid to the subtests: of -the McCarthy Scales 
of Children * s Abilities (Draw-A^Child, Verbal Fluency^ 
Verbal MeSpr^ and Arm Coordination) , Thesi efforts were 
reported/ in^nterim Report III^ Part A (March 1, 1976), ^ 
/The conckusion at that time was that since there were no 
apparent \4ifferences b'etween the results of McCarthy 
scoring procedures and alternative procedures, there would 
b^no advantage in adhering to the McCarthy^ ffcoring 
, conventions* " y 

However, re^exam^Mtipn^ of the psychometric dataifor 
the three "administrations of the battery ir:idicates a .ne.ed 
for revising this position v/ith raspeci to the Arm Coordination 
scale, .-When scored. using McCarthy criteria, internal 
consistency reliability (alpha) coef f icients ^ave been 
consistently lower than those , for other tests in the , battery 
(for the English^-dominant sample^ .34, ,62, and .65 for fall 
1975, spring 1976, and fall 1976). Since this s.cale is -^the 
on*ly' "pure'- psychomotor measure/ in the battery, 'the incentive 
- ttj^retain it is greater than it would be if it were redundant 
4With other constructs feapped by tha battery- Thus alternative 
scoring procedures ^erfe explo^Ped once againl - ' * 

' It was found that internal consistency; coefficients for 
the fall 1976 Arm Coordination data were ^romewhat higher 
when each of the six items in the scale as weighted, equally 
than , when they were weighted unequally ^ as they are in the 
McCarthy-' procedures . (Logic also argues for equal weighting 
since no argument is known to exist for unequal weighting.) 
Althouqh the internal consistency coefficient" for 'the English* 
dd^l^ant sample still just manages to reach the "nominal criterion 
t value of .65, it is judged to be sufficient to warrant: 
^ retention ^of this instrument, especially whpn consideration " ^ 
" is given to the^. relatively high test^retest '^correlation , .12, 
found iast year, ' . 

-7 : . . • 



J 
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APPENDIX D 
Flowcharts for the Analysis Procedure 
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H 
H 




lA, 

km any 
measures 
remaining to 
be analyzed? 



No 




ERIC . 



Figure D-1 

flow Chart for itep li Is'thi Measuri Reliable for This Sample?^ 



Yea 







Make tabli of 




response «rati 




by itim., Use 






for Box lE^, , 






i 




1 


B. 


Table 




measure 
aggrega 


3 for 
ite. 







ID. 

Are any items 
reniaining to 
be analyzed 
fQ)r this 
maasure? 




IE. 

Does item 
have rasponse 
rata less thaii 
90% in a^ 
categories? 



No 



IF. 

Bliniinata itein 
■ froi further 
reliability 
analyses; retain 
for othar 
analysis* 




T - Table for tiiis operation 
5 oparations 
^3 ^ Action decisions 
Q.GOtO ' 
^ Yas/No flows 
^ Mandatory flows , 





No 

L_._ 




/ 11*' \ 

Elimnata froni 


\ i 

\ ■ , / 


Aaasure those 




itaiiis whosa 




correlation with / ^ 


totil tast is 




Uess than JO, 








\ 

f 


IJ. 




Is original 




or yacc 


)rE|utid 


■ Yes 


Cronbac 
greataj 
,65? 


:h alpha 
' than 






No 
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1 



fete- 



Piguri'D-2 



Flow Chart testep j!. li^tiie'jfaaiuri Mid for This Sample?' 



H. 
H 





^ ■ ) 










■ t 




Ditaminf, ' ' 




Are 

: mi 




disirable r. 


—4* 


aris 


yal^ii ranges ,, 
,fgr all pairi 




'lift 




df iiaiures,^ 






j?za? 


■ , - ;-. .. 




1 

■ f / ■ 


No 










. \ ■ 

■1 . \ • • ■ 









., T' > Table fpr thii operatioft 
'P - Operationi' ; 
^Gtion dacision 
^- Go to , 
s Yei/No flws 
* Mandatpry flows 



4 
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4^. 



Ves ■ 



Do rvfluis ' 
fall within 
digirei ,v , 
rangas for / 
that'.iSiiurW 



2D. 

J^easure is 
valid for/ ■ 

Coipute total 
icora, 



■iff 






tmm 


fot. 


laek of ' : 


vilidit 






Sas ' ; 





1 



■p9 



■ v.- 



?Icv: Chart for Step- 3i 



° _ Figure D^3 r 
Are Raliability^*^^^r Validity Conitant Across Time and:^Samples? 









Detetmiiie 






, alphas for 






; fall 1975, 






Spring 1976, 






fall 1976, 













alph^'^t'^' 
to , confute? 




Table validity 
dm %B rmin a t i on 
for three 
time points. 











Are 


any 




^ires left 




fonipare? 





Evaluate reasons 
*f ot lack, of 
cqpip ar abi 1 i ty 



3C, 
Are alphas 
for a single 
measure 
^ompardble ^, 
across ' ttiree 
tdMe aoiints ? 



Yes 



3D. 
Measiire- 
considered 
reli^le 
for taxget ^ 
population . > 



No . 




3E, 

Evaluate reasons 
for lack of 
conparability 





3H, 

Is ^ val 2^1. ty . 






estimate 
cdn^ar^le ' 


^ Yes 




across three 
tiM, points? 






^No 





31. ^ ; 

Measure q6n-^^: 
sidered "valid 
and suitable 
for tatget 
op ul at ion* 
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. f~| m Operations 

K^J y « Action decisions' 

* = Yes/Ho ffows 



.7 
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FlQW Chart /for Step Dort ti|e Factor Structure Su^pQrt. Expectations ; 



hPae tor vainly all 
scales determined 
%p^p^ reliafale> 
'usifeg a prinoipar 
Gomponents solution 
(parform separate 
arialyses for Eng^ 
lish and Spanish 
samples) . 



Perform varimax 
rotation of the , 
factors obtained, 



4C. 

Det4rftine the 
CQmpQiitiyon of each 
£ aG4^r' inspect- 
ing 'the patte rn o f 
Jscale loadings; 
identify and la±)el 
the unifying 
cons tr ue tt^^ i 



^ operations 



r 



^ ^ ^ Action decisions 
- Go to^ 

^Mandatory flowe 




4D. 

Compare factor 
struature. for fell 
^ft76:;^to previous 
factor" structures 






4E , 

Cpmpare fall 1976 
factor ^struc€ures 

of English and 
?^panish samples . 
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Flow Chart for Step 5i ' 
Are the POC and Comparison Groups Comparable?. 



5A\ . 
Are any sites 
or aggregAtipns 
remaining to : > . 
be analyzad? "^^^ 



Ymm 





Compare EpC and com- 
parison groups^^i^ 
each site and 
sites oh eacljA vu,*^^, 
measure And^^h 
damdgraphiq^S^^^pJre , 
Perform comj^aris&nl 
separately for 
Enfgiish-- dominant and 
Spanish-dominant 
samples* Perform t 
tests or'x - as 
appropriate for 
metric and^categor^ 
ical variables 
respect ively. 




59. 

Evaluate the 
level of 

con^arability ^ ;^ 
of groups,/^)^^, . , 




ic 



^1 



^ operations 

- Action decist^ 

S^fio to ^ 



Yes/No flows-t- 
Mandatory flows 
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Figure D-6 



Flow ehairttWfor Step 61 Ars^Smple Sizes 
and Retention' Rates Adequate? 



"6 A.,, a 

rAre any sites 
"ot aggregations 
remaining to be 



Yes 




Of 'cur^;,^- 



^ Oh^lT^%^S ' o 

rent ea^le size and 
MtimatTO attrition^ 
project Bii#e of PDC 
and comparison grou; 
at each time'^oint ^ 
/critic all tQ_a pro- 
" spective longitudinal 




6C. . 

Evaluate 
implications 
for longlfcu-^ 
al study 4. 





f~| ^ Ogarations V 
^ .Action decisions 

, ^ = Yes/No flows 
- - - ^ Mandatory rffows 



123 
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APPENDTX E 



Magnitude , of pif ferencee- for Variables on 
Which Grojips Were ^lotand, Une(qual/ by/ Site 



r 




-X- . 



. Table E-l 
CaltfQjnia-^ritliih 



^lagpit^ Between PDG and Comparispn Analyt£ft 

^:!%^aR^lfer fot Vaiilables on Which Samples Were. Found Unequal (p<^r01) 



4 



■ ■■■■ \ ' 

BackgtrQund ; Characterl^ttfaB 



No. differen^s found, 



Test Perfbrmance^ 



No differences found', 



PDC 

(Nf37) ' 



Comparisbn 



K 



r 



V 
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Cali f ornia-^Spanish 



Magnlt-uda of Differences' Between PDC and Coinparxson^^naly^^ ' 
Samples for^ Variables on which S^mplae Were Found Unequal (p</J Ol) 



Background "^QHmtaaterlstiaa 
Np differences found. 



arieon ^ 



Test Performance 



Ho diffe^jpncee found. 



12 
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Table E-3 
Colorado 



Magnlit 



o£ DiEferances Between PDC and 



ison Analytic 



Sewiples for Variables on Which Samplee Were Found Unequal (p<, 01) 



Background Characteristics 
^ - . No differences found. « 



Test Perfor:mance 



PDC 
(NFir) 



CompariBon 



No differences found, 



Table E-4 
jnnecticut 



Magnituda o^Dif farencas Batwaan PDC and Comparispn Analytic 
Sampias for Variablas on Which Samples Were Found Unequal (p<,01) 



Background/ cfialipatatistiaB 

— ^— -fr^-— — 

No diflarancaB found. 



PDC 
(N^37) 



parlson 



Test Performance 



No differencas found. 





Table 



Georgia 



. Mag^tuda of Differewces Between PDC and Compmrison Analytic 
SajQp^-®^^ For Variables- on WhicM^Samples Were Found Unaqual .(p<,Oi) 




PDC 



No comparison group* 





Table E"6 
Florida 



Magnitude /of Dlfferencae Between PDC and Comparisbn Analytic = 
Sampiee tot Variables on Which^^amples^W^ .(p<.01) 



PDC 



Gomparlson 



Background Characteristics 
No differences found. 

Test Performance ^ 
POCL-2 



24 z 



Note . - 

Group means are given as £ scores*! A £ score eKpressds the 
difference. betwaen one group's mean and the mean ^or all caBes" 
in terms of standard deviation units Ce.g^*^ *50£ represents a 
position l/2» standard deviation above the overall mean)^ • 



ft 

Table 1^7 



Iowa 



,v>^gnitude pf Differences Betwaen PDC and Comparison Analytic 
'Samples fot' Variables on Which Samples Ware Found TUnequral * {p<. 01) 



Backgrourid CharacterlsfeioB 
No^ differences 'fcfiind. 



PDC 



Comparison 
. (N^51) 



Test Performance 

Verbal Fluency 
Verbal Memory^ 1 



^.332 



Note^ 



Group means are given aa ■£ scores* A score. eKpresses the 
difference between one group's mean and the mean £or all casei 
in tefinis of standard deviation units (e,g, ^ ' .SO^ represents ^a 
position 1/2 standard deviation above the* overall mean) , 




13 -J 
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. ^ / Table E-8 

*■* ' 

; .Maryland 

Magnitude of Differences Between PDC and Comparison Analytic 
Samples for VariableB on Which Samples Wete FOund Unequal ^(p<, 01) 



PDC , . Comparison 

^ , (N^2) / (NM5) 

. ■ . . f. 

Background Characteristics V.^ 
No differences found. j ^ 

Test Perforniance 

Verbal Memory =3 ,,AOz --.28z 



Note ■ ; ^^r^ 

Group " means are given as scores. A £ score expresses the 
difference between one group *s mean and the mean for all cases 
in terms of standard deviation units (e*g,, ,50£ represents a 
position 1/2 standard deviation above the overaXl mean) . 



Table E-9 



Michigan 



Magnitude of Differences Between PDC and. Cdntparison Analytic 
Samples for Variables on Which Samples Were Fouftd Unequal (p<, 01) " 



/ . 



Background Characteristics 

' • _ • ' ' ■ -J 

No differences found'* 



PDC 
|N^58) 



Comparison. 
, (N^S) 



Test Performance 



Verbal Memory-1 . 



-.23z 



24z 



Note 



Group means are giveft as £ scores, A £ score eKpresses the 
differ^ehce between one group's mean and the mean for ^aLl cases 
in terms of standard deviation units (e*g*; '^O^ represents 
position 1/2 'standard deviation above the overall mean)* 



- Table E-IO * \ ' 

■ Texas-Englisl^ 

Magnitude of Dif f erences-Setweefi PDC and Comparison Analytic 
Samples for Variables' on Which Sabples Were Found Unequal (p< . 01) 



* ■ ' I ' ^ ^ PDO Comparison 

^ . . ^ ^ . \^ (N^26) ;(N^19) ~ 

Backgrbund Character j,stics - ■ " - . 

/'No differences founH'* ' ' , ^ _ - 

Test. Performance . ^ ^ . ^ 

*No differences found* 




J ■ 




Table E-11 ■' / 

^ Texas-Spanish 



Magnitude of Differences Betwee#PDC and Comparison Analytic 
Samples for \fariables on ^Which Samples Were -Found- Unequal (p< , 01 ) 



i 



background Character is tics 



acteristics 



PDQ^ Comparigbn ^ 



Age . -.22m"( ..27£ 

Prior j^reschool 



Yes ^ ^ ^_ 3% 33% 

No ^ ... ^ ' 97% ' . &7% 



Test Performance , 

No differences fouftc 



Note 



Group means #are giyen. *as £ scores. ^ score expreas.es the 
difference between one , group ' s mean arid the^mean for /all caSes 
%n terms o^f standard deviation units (e.g./ * 5 0£ represents ' a 
position 1/^ standard deviation above the overall mean) J 



Table E-12 
Utah 



Magnitude of Differences Between PDC and Comparison An^^lytic 
amples for Variables on Which Samples Were Found Uneq,ual (p<.01) 



V 



Background Characteristics 
No differences found* 

Test Performance 
Height 



PDC 
(N^61) 



CompariBon , 
~ 1N^55) 



\ 



-.25z 



Note 



Group m^a|is are given as z scores. A z score expresses the- 
differenc^e between-^pne group's mean and the mean for all cases 
in tei-ms of standard deviation units (e.g., .SOz represents a 
position 1/2 standard deviation above the overall _jnean) . ■ 
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Table E-13 
Washington 



/ 



Magnitude of' Differences fietwaen PDC and Compariaon Arialytic' ^ 
Samples for 'Variables on Which Samples Were Found Unequal {p<.0^) 



' Backgro und C haracteristics 
No differenaes found 



Test Performance 



No differences found, 



PDC 
(N«49) 



\ Comparison 
f (Na66.) 



110 
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Table E-14 ' . ^= / 

' . % ' ' ' " . ' 

West Virginia * 

Magnitude of Differenqes Between PDC and Comparison Analytic 
•Sainples for Variables on Which Samples Were pound Unequal (p<p01) 



No differences found, 



PDC , Comparison 

(N^42) ^(N"^"29) 



Background Characteristics > 
_ — ^ r J 

No differences found. 



Test Performance 



r 



V 



J 



132 




- \ , ^ ■ ^ ' "^ ^^ V Table E-15 

English*Dominant Ag^gregate 

Magnitude' of Differenqes^ Between PDC and Comparisons Analytic 
Samplfes for Variables on Which Samples Were Found Unequal^ (p<*01) 

/ ' ' ' ' ■ 

^ ^ PDC Comparison 

Background Characteristics , . > 

( Nq differences' found. 

Test Performance 

Verbal . Memory-1 -.09z *102 " ' 



Note . ' ^ ^ 

Group means are given as^^ scopes, A. £ score expresses the- 
difference bet,ween^ one rgroup ' s mean and the. mean for all cases 
in terms of s€andard de^iition units (e,gp"^ .50^ represents a 
position 1/2 s^ndard deviation above the overall mean) . 




Tabf 



re E-16 



panlish-D^minant Aggregate 



Magnitude of Differences Between ^PDC and Comparison Analytic 
Saniples for Variables on Which Samples Were Found Unequal (p<*01) 



Test Performance 



PDC 



CompariBon 



, Background Characterlst: 

Prior Preschool EKp^rience 
( : Yes 
No 



11% 
89% 



39^ 
61^ 



No differences found. 



APPENDIX F 



PDC Classroont Observation Systetfi 
Definitions of Cate'gQries and i 
Means and Standard Deviations o£ Variables 



^^^^^^ 
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. 1 

Definitions of Observation Categories 
Category 1* N oninvolved i 




The child, is neither interacting wLth a 
object ^^ri^^ engaged in a purposeful activity^, 

Category 2* Invol^di 

- yi^ ' ^ 'U 

\ l^e child is amteracting with a person or object,! 

TMs category ds also coded when th% child is e^ag^ed 
in p purposefur activity such as singing to hiin/fter^ 

self. ; I 

Subcategory 2a > Focua of Attgntion i 

Social - ^ The child is engaged in a recip= 
rocal interaction with a peer 
and/or adult, 

Nonsocial % The child is ery^aged in a purpoee- 
ful activity wliich does not in= 
volve other, persons . 

Subcategory 2b. Language Spoken during Activa^ty i 

Verbal English i Wh^le engaging ffi social or 
■ ~' nonsocial activities , the 

child speaks only in English, 

Verbal Spanish i While engaging in social or 
" nonsocial activities, the 

c|iild ^peaks in English, 

Verbal Combined : While engaging in social or 
^ ^ nonsocial activities ; the 

child uses a conibination of 
• *' Spanish and English words', 

onverbal " While engaging in social or 



nonsocial activities , the 
child does not speak,- 



~ — I— 

=^For expanded definitions and examples of these categories ^ 
see Appendix E, Interim' Report IV , Part A (August' 1:, 1976) 
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Category^ 3 , Interactions with Peer ; 



The child engages in a repiprocal interaction with a 
]^eer(s) by loo^Eing at^ listening to, talking v/ith, 
or sharing materials and working on a^ convmon project, 



Subcategory 3a^^,.^.Descr iption of Peer In^teractiorf 



Negativ 



ContrA^ling 




7 

The child expresses verbal and/or^ 
physical aggression or hdstility 
toward the^peer(s) . 



Asserting 



Other ; 



The child a-^tempts, verbally or 
nonverbally , to direct, influence, 
ok manipulate the behavior of a 

r(s) in a positive manner. 
Thfe child's behavior is directed 
Qward on^ (or more) of the 
o-l lowing outcome changing 
t^e peer's course of ac^-On, ini- 
tiating a new behavior, showing 
(telling) the peer how to do 
something* . • 

The child does not comply with or 
ignores in a positive manner 
attempts made by a peer to control 
his/her behavior. 

The child, interacts in a cooper^ 
ative and^ positive manner with a 
peer(s). The child is sharing, 
helping , taking turns , working 
jointly, listening to, or talking 
with a peer, (This item is only 
marked^ for each positive behavior 
that is clearly not a controlling 
and asserting behavior.) 



Subcategories 3b and c . Purpose of Peer Interaction i 



Requests Information i 



The child, seeks (by posing 
a question, making a 
demand, etc.) a factual 
statement or eKplanation 
concernihg a task, profa-- 
lem, casual relationship, 
or other events op aitu-- 
ations in iis/hor eiwiron** 
ment. ') \ 
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Provides Information ; 



The child offers in-- 
formation in the form 
of factual statements, 
explanations , or physical 
gestures, . ^ 



Requests Assistance : 



P^vides Assistance % 



Requests Support' i 



The child seeks physical 
assistance or materials 
for initiating or com- 
pleting an activity* 

The child' of ters physical 
assistance or materials 
for^ initiating or com- 
pleting an activity. 

The child' seeks comfort, 
protection , or reassurance 
after a hurt, disappoint- 
ment, or other problem 
situation* ^ The child does 
not request assistance or 
information for solving 
the problem. 



Provides Support : 



Nonapplicable : 



The child provides a peer 
with comfort, protection, 
or reassurance after a 
hurt^ disappointment, or 
other problem situation - 
The child does not provide 
aseistance or information 
for^ solving the problem* 
This itpm is also coded 
5pr vecbal expressions of 
syqg^hy or empathy. 

The puj^ose of the child's 
interaction with a peer is 
clearly not one of re- 
questing/providing infor- * 
mation, assistance, ma- 
terials , or emotional 
suppor^t. 



Category 4, Interactiohs with A dults i ' 

The child engages Jin a -reciprocal interactiori with an \ 
adult (s) by iookiM'at^ listening to^ talking with ^ 
or sharing materirfs and" working on a conmion'^ project, 

^ ■ . - / ^ ^ ' ^. 

^ . Subcategory 4a > . Type of Adult Interaction i 

The child eKpreases verbal and/or 
physical aggression or hostility 
toward the adult (s) , * ' 

*"* ■ ' 

Controlling I The child attempts^ verbally or 

" nonverbally^ tb direct^ influence^ 

or manipulate the behavior of 
• . an adult(s) in a^ positive manner. 

The child* s behavior is directed , 
^ toward one (or more) of the 
following outcomes i changing ^ 
the adult's course of action^ 
_ , ^. .initiating a new behavior ^ or 

showing ( tej.lin^) th# adult how 
to do something* 

Asserting i The .child does not comply with or 
ignores in a positive manner ^ 
attempts made by an adult to 
control his/he^ behavior * 

O ther I The child interacts in a coopera-- 

~ ' tive and positive manner with an 

adult (s)* The child is sharing, 
• helping, taking turns, working 

jointly, listening to, or talking 
with an a^dult. (This item is . 
* only marked for each positive 
behavior that is 'clearly not" a 
bontrolling and asserting behavior) 

Su bcategories 4b and c. Purpose of Adult Interaction : 

R equests Assistance i The^ chiid seeks physte^ 

assistance or materials 
= \ f 6r initiating o?: .com-^ 

' pitting an aotivit^'' 




Provld-as Assistance i ; The child offers physical ^ 
" ' " assistance or materials 
for initiating or coin- 
pletiflg. an activity. 



Requesti. Support s 



7 ^ 



/ Provides Support s 



. Nonapplicable g 



r5 



child seeks comfort^ 
)tection/ o^r reas- 

im after a hurt> ^ 
.sappointrnant , or other 
problem situation. The 
child does not request 
assistance or informatidn 
for solving the problem. 

The' child provides an 
adult with comfort/, 
protection^ or reassurance 
after a hurt, disappoint- 
ment, or other problem 
situation. The child 

oes not provide 'assistance 
or information for solving 
the probiam. This item is 
also coded for verbal 
expressions of sympathy or 
empathy, " 

The purpose of tha child's 
interactibn with an adult , 
is clearly ^ not one of 
requesting/providing 
information, assistance, 
materials, or emotional 
support. 



Ca^^tggbry 5, ^Classroom Activity Lev^l i 



This category describes the opportunity ' children have 
to spontaneously engage in or initiate interactions 
^ with others. The observer's attention is, no longer 
directed toward the target child, but toward the 
Qlassroom as a whole. After surveying the classroom, 
one of the following items is coded after each 
5- second observation interval i 
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Maximal I This reJari to those parts of the day in 
whiGh children and adults ars free to 
initiate or maintain spontaneous inter- 
V V , actionB (verbally or physically) . among 

j \ ' • themselves. The children* are generally 

V- -J.' able to choose their own activity^ with 

^ ■ minimal gtruc turlng or directiiDn by an 

V. .adult. Teachers sometimes aall thase 

" \ ' ' "free play'*''or "free choice" periods. 

Moderate : This refers to those parts * of the day in 
~ which the opportunity for spontaneous^ 
" ^ ' ' interaction among adults and children is 

. ^ . . substantially^ reduced. During this 

- period^ classroom behavior is typically 
^ decided less , by children than by adult 

' direction,^ There is still some oppor- 

tunity for spontaneous interactions to 
^ccur within this structure. 



Minimal i This refers to those parts of th« 4ay in 
which children ar^e not free to initiate/ 
maintain spontaneous interactipnB (verbal 
or physical) among themselves , Class- 
room behavior of the children is primarily 
controlled and directed by an adult. 
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Table , P-1 



Meani and stand 



ard Diviations of Classroo|^CJiild-Adult interactions* 



Obsnrvition Viriabli 



idult Interacti'Ons 



Description of 



Purpose of ChiM-Adulu Interactions 

■GinMhUtdm/fktihlaU 



ACROSS Mil 
ittivlty 
levili 



Mean 



.02 
.65 



.08 

.55' 
.13 
K .07 

0 



,20 ' 

.05 

.20 



.07 
■.19 
,11 
.12 
0 
0 



activity 
leveli 

^ - ( 



Maan i.D. 



o 

M 
.02 
.58 



,11 
.51 

\m 

.07 

0 

0 



0 

.30 
.06 
.30 



1 13 

^0 ■ 
.12 I 

0 



MODEMTI 
astivity 
lavali 



Mean S.Di 



0 

.36 

.OS 

.63 



.08 

.5^ 

.lit 

.06 

0 

0 



{) 
.28 
.06 
.27 



,11 
.27 
.19 
.Ik 

0 
'0 



MINIMUM 
activity 
iiVilg 



Mean 



0 

.21 

.'OS 
.77 



.or 
.08 

(y . 
0 



0 
.26 
.06 
.'26 



.07 

.35 
.19 

.26 

0 

0 



Estimates 'repregent relative frequencies. 



Relative • friquency of- this category fell between .00 and .01. 



Table w2 



°- 



lObsimtion Varltblt 



Clagsrooffl Involvement 

■ Honimolvid 
. Sc 



Classroom Verbal-* Behavior 



Miani and Standard Deviations &f .Classropm 
Involvement and Clasiroorn Verbal Bihavior, Varlablii* >'.• ^ 



ACROSS ALL . 
aetivity 
liveli 



Maan 



.06 

M 
M 



.2 



7 
,03 

JO 



.07 
.16 
.16 



.oa 

.08' 

() 
.08 



activity 
levels 



Mean i i.D. 



.05. 
.kk 

.51 



.29 
■.03 
0 
.69 



.10 
.19 
.19 



.13 
.08 

() 

;i3 



DDIRATE 
, Aetivity ■ 
levili 



Heah 



,■07 
M 



.25 
.03 
0 
.72 



-.08 

■.19 
.18^ 



{ 



.10 
',09 
0 
.11 



i 



MINIHUM 
ictivity 
levels 



Mean 



.08 

,6k 
.28 

A 



.19 
,03 
() 
.77 



.07 

0 

.15 



Estimates represent ralatlve frequencies. 

^^cD?/-' frequency of tMs category fell between .00 and .01, 
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M,0ans:an^'^Tidard Daviat'ibns of .Cl'asarooin Child-fier Interactions 



a 



Tbiirvatlen /variabli 



Pescr.iptiori df Child-Peer Interactiofls 



Purpose of Child-Peer Interactions 
Gin Uiofmation' 
Reqaei-t SiippoU 



ACBOgS ALL 
Jiotivity 
Uivili 



Msan 



if 
S 
.Q3 

58 



.08 
M 
.17 
.10 

0 
0 



0 
M 

.22 



.0? 
.18 
.11 
.12 
0 

0 



levilg . 



.07 
.1*0 

.55 



.08 

.20 
.11 
(.) 
() 



.01 

.08 
.25 



.01 
.20 

.15 
'.16 

0 
0 



levjils 



Mean 



0 
.37 
.02 
.61 



,11 
.52. 
.Ih 
,10 
0 
■()■ 



.0 

-03 



r=^;i6 
''^.23 
.15 
.15 
0 
0 



L 



. M1N|MUM 
aefrivlty, 
iiVils 



Mean 



.03 
.36 
.0 




.05 

M' 

.'20 
-09 

0 



Ml 



.12 
.36 
.12 

.36 



if- • i ' ■ ^ ■ 

Ei^imates represent relative frecjuencie 

i , 

Relative frequency of this category fell betweeh .00 and; .01. 
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APPENDIX G 



ibscales of the POCL' 
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. 7 • ^ - Subscales of the I^OCLl 

^ f - V ^ ■ ■ ' i ' " . \ . . " .. 

' Factor analysis^ of the Pu^iU Observation l^|e^^ . : 

yialded t^a>fdistinct fac'tqpe (accounting for 47,6% in^ j 
31*4% of the^ variance / £%spectivelyr w composition . — ' 
is 7#s0sntially identicaa to that of the factors discovered 
in* past^ analyses of the POCL.v' As before, they^h^va been 
named "'*iTask Orientatiop" and "EKtroversion" to deeorSie 
the common ciiaracteristics of their constituent items. 
The loadings of each item on each factor are shown in 
Table G-1, Scores on the subscales were calaulated by 
summing the actual scores on all items for a'subscale. 
These subscale. scores wfer© then used for sub^equenj; analyse^ 
Of the pGCL fall 1976 data. * ^ * ' . . i 

Psychometric analyses of spring 1976. and fall 197^6 
^OCL data indicate that the two subsca:ias possess a h^h > , 
degree of internal consistency (a >"*90 at both time-^ ^ 
points). Since this instrument is used 'by testers at all 
sites^ for all groups^ scores on thesa scales could be, /% 
coi^sldered equivalent for Eriglish-dominant an4 Spanish- 
dontin^t samples and might enter usiJully into iuture^ • 
analyses^ of data pooled across language groups. 



-principal components solution/ varimax rotation. ' 
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, ' ■ . ... Table- G-X ■. ' 

sscales o5 the POCL, Based o^Facftdr Analysis* ' 
ofy^tem Scores 'for All Children ils' the Analytic SairtpXe 

Fall 1976 :Da'^ 



N«91$ 



\ , / 


Loading of Item on Each* Factor; , . 
.^(higher loading italicized) 


: : - 


Fa9^or 1 ' 

'"Task Orientation" 

■ . - . ■ -■ > 


■' Factor 2 

"Extroversion" 

- > - 


Cpop^rktive \ \ 
InvoLved * 


: ... , 


.40- . 
.55 ( 


Talkative 


.32 - 


" .B7 . 


Attentive/* 


.S3 


.26 . 


Active / ^ 


.25 






,76 


• .39 


Quick to respond ^ 


.82 


.52 


Attemptp difficult 
tasks 


.81 


.41 


Keeps trying 


.86 


. 29 


Realistically self ^ 
confident 

1 


.89 


.21- 



Principal components solution , varimax rotation. 
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APPENDIX H 



Analytic. Procedure fbr Investiwation of PPC and 
Comparison Classroom Comparability on whfe Observation Variables 
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Analytic Procedure ^or Inyestigation of PDC and 
Gomparison Classroom Comparabili^ on the Observation Variables 

When analyzing data that represent proportions (such 
as the Relative frequencies in the PDC Observation System) -, 
, the data are sometimes transformed mathernatically (e*g, , 
arcsiTi of rfe square root of the proportldn) to "normalize" 
■ the distributions^ of the prbportion. ^^hese transformation's 
are^ased on the assumption that the characteristic in 
question is distributed normally in thii larger target 
^population and thus should be forced lAto ^ normal config- 
' uration for the sample* For. various reasons/ that assumption 
is not tenable for thefee d^a; Consequently, analyses of 
PDC and comparison classroom comparability were performed 
using untransf ormed relative frequencies of the observation 
- variables . ^ 

Since the purpose of performing these comparability 
analyses -was to test the assumption that PDC and comparison 
group classroom means would not be significantly different, 
a liberal significance level of .10 was lelected as the 
criterion for evaluating differences, T^©'%group. variables 
were created for these comparisons* In the first, all 
classrooms containing any' children^ tested in English were , 
split into PDC and comparison groups. In the second, all 
classrooms with any jchildrgn tested in Spanish were divided ^ 
into PDC and comparison groups. 

An initial of analyses was pel' formed Yon the \ 

relative frequencies using the parameta^ic t^^es^ This « 
statistic is robust with respect to violatio^^ ot\ 
normality assumptions. However, in the event ItLax the 
difference between gjoup variances is significMit, the 
results of such anaryses must be interpreted cautiously* 
'Consequently, a second set of non.^parametric analyses, 
using the median test, were also performed forV^e same 
English-dominant sample (PDC vs, comparison) and^panish- 
dominant sample (PDC vs * comparisori) * 



